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Preface 



This volume contains the contributions to the Joint German/ Austrian Confe- 
rence on Artificial Intelligence, KI 2001, which comprises the 24th German and 
the 9th Austrian Conference on Artificial Intelligence. They are divided into the 
following categories: 

— 2 contributions by invited speakers of the conference; 

— 29 accepted technical papers, of which 5 where submitted as application 
papers and 24 as papers on foundations of AI; 

— 4 contributions by participants of the industrial day, during which companies 
working in the field presented their AI applications. 

After a long period of separate meetings, the German and Austrian Societies 
for Artificial Intelligence, KI and OGAI, decided to hold a joint conference in 
Vienna in 2001. The two societies had previously held one joint conference. 
This took place in Ottstein, a small town in Lower Austria, in 1986. At that 
time, the rise of expert system technology had also renewed interest in AI in 
general, with quite some expectations for future advances regarding the use of 
AI techniques in applications pervading many areas of our daily life. Since then 
fifteen years have passed, and we may want to comment, at the beginning of 
a new century, on the progress that has been made in this direction. Although 
significant advances in AI research and technology have been made within this 
period, we are still far from having reached visionary goals such as, for example, 
the capabilities of HAL, the super computer, in Stanley Kubrick’s famous film 
“2001: A Space Odyssey,” let alone Spielberg’s interpretation of Kubrick’s more 
recent expectations regarding the future of AI. In this respect, AI is not yet 
as good as Pinoccio’s blue fairy, but easier to find: just go to the annual KI 
conferences. 

The goal of this joint conference was to bring together AI researchers wor- 
king in academia and in companies, and let them present and discuss their latest 
research results, both on theoretical foundations as well as on applications; these 
are the two legs any healthy field needs to stand upon. As we can see from the 
contributions to this conference, AI appears to be in good shape in this respect. 
In particular, during the Industrial Day of KI 2001, we could observe a number 
of exciting industrial applications of AI techniques in areas as diverse as con- 
figuration, elevator control, supply chain management, and speech recognition. 
These applications ~ and many others the general public and sometimes even AI 
researchers are less aware of - confirm the fact that AI technology has made its 
way silently into numerous applications (and will certainly continue to do so) . In 
the emerging information society, AI techniques will play a key role for intelli- 
gent systems that remain to be built (such as really intelligent search interfaces 
for the Web). In this respect, the research presented at the conference is encou- 
raging, and makes us confident about the future prospects for and developments 
in the area. 




VI 
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Following the trend of recent German Conferences on AI, this conference 
turned out to be not just a local event for AI researchers from Germany and 
Austria, but an international conference that is of interest to researchers from all 
over the world. This was reflected in 79 submissions from a total of 22 countries: 
Algeria, Australia, Austria, China, Cyprus, Finland, France, Germany, Greece, 
Iran, Israel, Japan, Lithuania, Romania, Slovakia, Spain, Sweden, Switzerland, 
Taiwan, The Netherlands, UK, and the USA. Of these submissions, 15 were 
submitted as application papers and 64 as papers on foundations of AI. Of the 
29 accepted papers, 3 were quite outstanding in the sense that they clearly 
obtained the best grades by the reviewers. These papers are grouped under 
the heading “Selected Papers” in the proceedings and they were presented in 
a special session (without parallel sessions) at the conference. From these three 
papers, the program committee selected the paper 

“Approximating Most Specific Concepts in Description Logics with 

Existential Restrictions” by Ralf Kiisters and Ralf Molitor 

for the Springer Best Paper Award. Congratulations to the authors for this 
excellent piece of work! 

A large number of people helped to make this conference a success. Our 
thanks go to the workshop chair (Jurgen Dorn), the industrial chairs (Gerhard 
Friedrich and Kurt Sundermeyer) , the local arrangements chair (Uwe Egly), 
and all the other people involved in the local organization in Vienna. Special 
thanks go to Elfriede Nedoma, the conference secretary, whose continuous efforts 
were vital to the whole enterprise, and to Wolfgang Faber, who maintained the 
marvelous web-site of the conference and designed posters, folders, and other 
important items. Thanks Wolfgang, you would be a fabulous artist! 

As for the technical program, first and foremost we thank all the authors for 
submitting their papers to our conference. Secondly, we thank the members of 
the program committee as well as the additional reviewers who did a tremendous 
job in writing reviews and participating in the electronic PC meeting. Their effort 
made it possible to select the best papers from the large number of submissions 
in a relatively short period of time. 

Next, we would like to thank our distinguished invited speakers Hans Kamp 
(University of Stuttgart), Michael Kearns (Syntek Capital), Raymond Reiter 
(University of Toronto), and V.S. Subrahmanian (University of Maryland) for 
kindly accepting our invitation to give a talk at our conference. Furthermore, we 
thank our tutorialists, Thom Friihwirth (University of Munich) and Stefan Wro- 
bel (University of Magdeburg) , for their lectures that made attendants familiar 
with developments in the areas of constraint handling rules and data mining. 

In order to obtain funding for invited speakers and other important events 
without having to take outrageous conferences fees from the participants, it is 
vital to obtain support from industrial and other sponsors. Our thanks in this 
respect go to the following companies: Microsoft, Siemens, Springer- Verlag, and 
Sysis, and the following institutions and organizations: the European Commis- 
sion, the Austrian Computer Society (OCG), the Austrian Economic Chamber, 
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the Austrian Ministry of Transport, Innovation & Technology, the Austrian Mi- 
nistry of Education, Science and Culture, and the City of Vienna. 

Finally, we would like to thank Carsten Lutz (RWTH Aachen) for installing 
and managing the electronic system (ConfMan) that allowed us to get all sub- 
missions via the Internet, and to have a virtual PC meeting. You did a great job! 
Carsten Lutz together with Ulrike Sattler helped to produce the camera-ready 
copy of these proceedings. 

September 2001 Franz Baader 

Gerhard Brewka 
Thomas Eiter 
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Michael Kearns 

Syntek Capital, 423 W. 55th Street, New York, NY 10019, USA 
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Abstract. There has been growing interest in AI and related disciplines 
in the emerging field of computational game theory. This area revisits the 
problems and solutions of classical game theory with an explicit emphasis 
on computational efficiency and scalability. The interest from the AI 
community arises from several sources, including models and algorithms 
for multi-agent systems, design of electronic commerce agents, and the 
study of compact representations for complex environments that permit 
efficient learning and planning algorithms. 

In the talk, I will survey some recent results in computational game 
theory, and highlight similarities with algorithms, representations and 
motivation in the AI and machine learning literature. The topics ex- 
amined will include a simple study of gradient algorithms in general 
games 0, the application of reinforcement learning algorithms and their 
generalizations to stochastic games [ 2 |, and the introduction of compact 
graphical models for multi-player games m- Interesting directions for 
further work will be discussed. 
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Abstract. The Internet contains a vast array of sources that provide identical or 
similar services. When an agent needs to solve a problem, it may split the problem 
into “subproblems” and find an agent to solve each of the subproblems. Later, it 
may combine the results of these subproblems to solve the original problem. In 
this case, the agent is faced with the task of determining to which agents to assign 
the subproblems. We call this the agent selection problem (ASP for short). Solving 
ASP is complex because it must take into account several different parameters. For 
instance, different agents might take different amounts of time to process a request. 
Different agents might provide varying “qualities” of answers. Network latencies 
associated with different agents might vary. In this paper, we first formalize the 
agent selection problem and show that it is NP-hard. We then propose a generic 
cost function that is general enough to take into account the costs of (i) network 
and server loads, (ii) source computations, and (iii) internal mediator costs. We 
then develop exact and heuristic based algorithms to solve the agent selection 
problem. 



1 Introduction 

There is now a growing body of work on software agents. Several impressive platforms 
for the creation and deployment of agents have been developed II/II 11121 . In this paper, 
we focus on the agent selection problem (ASP) for short). 

Suppose we have an agent that provides a set of services. Such an agent may need to 
access a whole slew of secondary agents to provide its services. For instance, an agent 
associated with a transportation company may provide shipping services to external par- 
ties, as well as provide detailed routing and shipping manifest instructions to its truck 
drivers. In this case, the transportation agent may obtain routing information from 
mapquest and mapblast agents. Likewise, the company’s truck drivers may obtain appro- 
priate yellow pages information (e.g., restaurants on the route) from third party servers 
like yahoo and verizonyellowpages. Informally put, given a current service S being 
provided by an agent, the agent selection problem says how should we select agents 
to handle the individual atomic services in S so that some performance objective is 
optimized? 

The answer to this depends on a number of complex factors. First, certain sources may 
be better for a given service than others — for example, an agent that warehouses data may 
be “fast” compared to an agent in Indonesia, but the Indonesian source may contain more 
up to date information. Second, certain agents may have a “heavy” load at some times. 
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but not at others. For instance, it may turn out that cnnfii is much more heavily loaded 
than financiall.com at certain times of the day - so at those times using financial! .com 
for stock ticker information may be better. Third, even thoughfinanciall.com may have 
lower load at certain times, it may turn out that congestion points on the network outside 
the control of financiall.com cause network access to it to be very slow. Fourth, even 
though financiall .com may have a lighter load than cnnfn, its response to requests may 
include lots of advertisement objects which consume network bandwidth and buffer 
resources. 

All these factors make it clear that given a service S that an agent wishes to pro- 
vide, assigning agents to handle appropriate parts of S depends not only on the agents 
themselves, but also on other factors such as the network, extraneous data returned by 
the sources, freshness of data in the sources, and of course, how well the agents perform 
their computations. 

In this paper, we first formalize the agent selection problem (ASP) and show that it 
is NP-hard. Second, we propose an architecture which uses a “generic” cost function — 
this is very general and hence, we may plug in different cost models for individual costing 
of (i) network operations, (ii) source computations and (iii) internal agent computations, 
and use the cost function to merge them. Third, we provide algorithms that: given a 
service S will produce a way of assigning agents to the atomic services in S and propose 
a way of ordering the atomic services of S so as to optimize some performance criterion. 

2 Preliminaries 

We assume that every agent provides a set of services and that agents may be built on 
top of legacy databases, data structures, and software packages. An agent has a set svc of 
primitive service names. Each service s G svc has a signature specifying the inputs and 
the outputs of the service. For example, we may have a service called directions which 
has inputs addrl/addrtype, addrl/addrtype, which takes two addresse£| as input and 
returns a set containing a single list of strings as output (specifying driving directions) 
if it is possible to drive from the first point to the other, or the empty set if no such route 
can be generated. It is entirely possible that the same service (e.g., directions) may be 
offered by many different agents. 

We assume without loss of generality that all services return, as output, a set of 
objects. As usual, we also assume that each data type t has an associated domain dom{T) 
— a term of type r is either a variable ranging over r or a member of dom(r). If X is a 
variable of record type r and fi : is a field of r, then X.fi is also a variable. Likewise, 

if Ti is itself a record type with field fj : tj, then X.fi.fj is also a variable. Variables 
like X.fi and X.fi.fj are called path variables and in this case, their associated root 
variable is X. In general, we use root{V ar) to denote the root variable associated with 
Var. Moreover, if A is a set of variables, then root{A) = {root{X) \ X G A}. 

Definition 1 (Service atom). Suppose s is a service having input signature (ri , . . . , ) 

and ti is a term of type Ti for 1 < i < n. Furthermore, suppose the output type of s 
is {r} and t is a term of type r. Then: in{t, s{ti, . . . ,f„)) is a service atom. 1ft is a 
variable, then it is called the output variable of the above service atom. 

* addrtype may be a record type specifying a number, street name, city, state, zip, and country. 
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Convention: In this paper, we use lower case letters to denote constants and upper 
case letters to denote variables. 

We assume the existence of an agent called service directory agent (SDA) which 
may he queried by other agents seeking to know which agents provide which services. 
There are many ways in which SDA may be implemented IMl- In this paper, we do 
not go into mechanisms to implement SDA. For the sake of simplicity, we will assume 
that SDA has a table of services and agents that offer them. This assumption is not 
necessary - it may be easily replaced by assuming that an existing implementation of 
SDA is available and is used. Table Q] shows a list of services that may be offered by a 
multiagent application dealing with traffic information. 



Table 1. Services offered by the Traffic Information application 



Service Name 


Agent(s) 


directions (addressl , address2) 


mapquest, mapblast 


getmap ( addr e s s ) 


mapquest, mapblast 


status (city .highway)^ 


smartraveler 


status (city .highway 


etaktraffic 


getinfo (category, name, city, state) 


yahoo,verizonyellowpages 


sqlstring 


facilitiesdb 



Users of the traffic information application might want a variety of actions taken on 
their behalf by agents. Two representative examples are given below. 

1 . “At least 45 minutes before a scheduled departure, find driving directions to desti- 
nation X from current location y so that there are no congested roads along the way 
and print them out.” 

2. “If the expected arrival at destination x is between 1 1 :30 am and 1 ;30pm or between 
6pm and 8:30 pm, then find the address and the phone number of the closest restaurant 
to address x and print it out.” 

A user may request the agent to monitor properties of this type for him. It is important 
to note that these properties are rules of the form: 

Head •<— Body 

where Head is some type of action to be taken, and Body is a condition involving 
requests (possibly) to third party agents. In the second case above, the request might 
involve contacting a yellow pages agent (e.g., verizonyellowpages or yahoo) to obtain 
a restaurant listing, and then a subsequent request to a map agent (e.g., mapquest or 
mapblast) to provide directions to a selected restaurant. The Body part of the above rule 
is called a service condition defined below. 

^ This returns a string, e.g. “no congestion”, “slow”, etc. 

^ This returns a list of strings, reflecting incidents on the highway 
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Definition 2 (Comparison atom). Suppose t\,t2 are terms over the same domain. Then 
ti = t2 and t\ ^ t2 are comparison atoms. In addition, if the domain involved has an 
associated partial ordering <, then ti < t2,ti < t2,ti > t2,ti > ^2 are comparison 
atoms where >, < are defined in terms of>,< and equality in the usual way. 



Definition 3 (Service condition). A service condition is defined inductively as follows. ■ 
(i) every service atom and comparison atom is a service condition, (ii) if XitX^ are 
service conditions, then so is xi A X2- 

For example, the service condition 

in{H, getinfo{hospital, nil, bethesda, md)) A in{D, directions{addrl, H.addr)) 

expresses the condition that we want to find a route from addr 1 to a hospital in Bethesda. 
To evaluate this service condition, one might either use Verizonyellowpages or Yahoo 
to answer the getinfof) condition and either Mapquest or Mapblast to compute the 
directions{) condition. Which combination should be chosen depends on a number of 
parameters including network traffic, historical data on the behavior of the servers, etc. 

Even though the SDA may say that two or more agents (e.g., Mapquest and Mapblast) 
provide the same service, this does not mean that all these agents provide the same 
number of answers, nor does it mean that these answers will necessarily be the same. 
For instance, some sources may be updated in real time, while others might be updated 
less frequently. Some sources may add objects (e.g., advertisements) to their answers, 
while others may not. When two or more agents are listed by the SDA as offering the 
same service, we will assume that the requesting agent is indifferent to any differences 
in the answers returned by these two agents. 

Throughout the rest of this paper, we assume the existence of some arbitrary, but 
fixed SDA that is used by other agents. 



3 The Problem 

In this section, we describe the technical problem that we solve in this paper. We start 
by providing some basic notation. 

Suppose X is a service condition. We use Atoms{x) to denote the set of all service 
and comparison atoms in x- A service condition x' is said to be a subconjunct of x iff 
Atoms{x!) ^ Atoms{x)- A service condition partition (SCP) of x is any partition 
of Atoms{x)- Intuitively, an SCP of a service condition x splits x into subconjuncts 
Xi , . . . , Xn so that each Xi can be serviced by a single agent. 

Definition 4 (Sourced Service Graph). Consider an agent A and suppose x A a service 
condition. A sourced service graph (SSG for short) associated with x is a directed acyclic 
graph {V, E) where: 

— V is a partition of x and 

— Every vertex in v is labeled by the name of an agent recognized by SDA. 
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Fig. 1. Example SSG 



Fig. 2. Example SSG 



A sub-SSG associated with x ^ directed acyclic graph (V, E) where V C 
Atoms{x)’ tind every vertex is labeled by an agent recognized by SDA. 

Note that for now the above definition poses no restrictions on the edges in E (other 
than acyclicity). Later, (Definitions 0and0 we will impose some constraints on SSGs. 
Suppose we have a service condition x and a partition V of x- 

Two examples of sourced service graphs are shown in Figures n]and|3 It is important 
to note that a vertex is a set of service atoms and comparison atoms. By taking the 
conjunction of these atoms, we can associate a service condition SC{v) with the vertex 

V. 



Example 1. The service condition 

in{P, getinfo{post — office, nil, rockville. State)) A 
in{D, directions{addrl, P.addr))A 

in{S,status{rockville,D. highway)) A S = non — congested 

finds a non-congested route to a post office in Rockville. Figure Eshows an SSG for it. 
The service condition 

in{List, sqlstring{“select name, dist from facilities where 
city = Rockville' and type = hospital”)) A List. dist < 5 A 
in{H, getinfo{List.name, nil, rockville, md))A 
in{D,directions{addrl,H.addr)) A P[ .burnunit = true 

asks for driving directions to a hospital in Rockville which is at most 5 miles from 
downtown and has a burn unit. The SSG for it is provided in Figured Note that in this 
SSG facilitiesdb agent handles a conjunction. (Agents such as those in the IMPACT 
system uni can easily handle conjunctions even if the agents are built on top of data 
sources which cannot. Hence there is no loss of generality in making this assumption). 

It is easy to see that a sourced service graph may not “make sense.” For this to happen, two 
conditions must be satisfied. First, if an agent labels a vertex, then it must be possible to 
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execute the service condition labeling that vertex by using just that agent alone. Second, 
the service condition should be executable, i.e., we should somehow be able to guarantee 
that all the variables occurring in that service condition which need to be instantiated 
for it to be executable will in fact be instantiated. For instance, the service condition 
in{D, directions{addrl, H.addr)) can be handled by Mapquest (which provides the 
“directions” service) but to execute it, H. addr must be instantiated. These two conditions 
are captured via our notions of a “feasible” sourced service graph and an “evaluable 
graph” as defined below. 

Definition 5 (Feasible SSG) .An SSG {V, E) is feasible ijf whenever a vertex x E labeled 
with agent A, it is the case that every service atom of the form in(t, s(. . .)) is such that 
s is a service offered by agent A according to SDA. 



Example 2. The SSGs in Figures Q] and 0 are both feasible with respect to the ser- 
vice directory given in Tabled Both Mapquest and Mapblast offer the directions ser- 
vice. Similarly, both Verizonyellowpages and Yahoo provide the service getinfo. Finally, 
Smartraveler offers the status service. 



Definition 6 (Evaluable SSG). An SSG (y, E) is evaluable iff for every vertex v in V, 
there is a permutation Xi A . . -AXn of its atomic constituents such that for all 1 < i < n: 

ff Xi = in(t, s(ti , . . . , tm)) and E is a (root or path) variable then either: 

a) there exists a vertex v' in V such that there is a path from v' to v and there exists 
either a service atom x'i in v' of the form in{root(tr) , s' {. . .)) or a comparison 
atom x'i in v' of the form rootlff) = term where term is either a variable or 
constant, or 

b) there exists a j < i in v such that either Xj E of the form in(root(tr) , s' {. . .)) 
or of the form rootftf) = term where term is either a variable or constant. 

2. if Xi E of the form t\ = t 2 , then either at least one oft\,t 2 E not a variable or 
either t\ or t 2 satisfies condition (a) or (b) above; 

3. if Xi E of the form tiopt 2 where op is one of <, <,>,>, then each oft\, t 2 must 
satisfy either condition (a) or (b) above. 

Moreover, a vertex x in V is evaluable iff all (root or path) variables E appearing 
in X satisfy condition (a) or (b) above. 

Example 3. The SSG given in Figure's not evaluable because the atom in{P, getinfo{ 
post — office, nil, rockville, State)) does not satisfy the above conditions. In partic- 
ular, there is no way to instantiate the variable State. On the other hand, the SSG of 
Figure Qis evaluable. This is because every vertex (except for the first one) contains a 
single atom and each of the atoms satisfies condition (a) of Definition 0 and the first 
vertex satisfies condition (6) of the definition. 

Definition 7 (Cost function). A cost function is a mapping, cost, which takes as input 
a feasible, evaluable SSG for a service condition x tmd returns a real number as output. 
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Intuitively, cost{V, E) measures how expensive it is to evaluate the SSG. Due to space 
restrictions, we do not, in this paper, go into details of what the cost model associated 
with an agent looks like. The full version of this paper will develop extensive network 
based cost models for this problem. Suffice it to say that by merely requiring the ex- 
istence of a cost function that maps SSGs to real numbers, we are allowing ourselves 
the option of plugging in any kind of cost model whatsoever. Such a cost model will 
consist of various parameters. These parameters will include ways of measuring: (1) es- 
timated network time, (ii) estimated computation time by remote agents, (iii) estimated 
cardinality of the set of answers returned by the remote source, (iv) estimated size, in 
MBytes, of the answer, (v) estimated freshness of the answer, measuring the accuracy 
of the answer. Based on these parameters, the cost model will combine such values into 
a single composite value. 

Building on top of database query optimization methods, there has been considerable 
research performed to date into how we may optimize evaluation of a set of service 
conditions lllbi;-ill . All these methods assume the existence of a set of rewrite rules that 
allow a service condition to be rewritten into many equivalent, but syntactically different 
forms corresponding to different ways of evaluating the service condition. Given a set 
RR of rewrite rules for an agent, let RewRuix) be the set of all possible rewritten 
versions of service call %. 

We are now ready to define the agent selection problem. 

Definition 8 (Agent Selection Problem (ASP)). Suppose A is an agent and RR is a 
set of rewrite rules. The agent selection problem (ASP) is: 

INPUT: Service condition rewrite rules RR and cost function cost. 

OUTPUT: Find a feasible, evaluable SSG (V, E) associated with a member of 
RewRii(x) ^tich that cost(V, E) is minimized. 

Our first result below is that ASP is NP-hard. The proof constructs a reduction of the 
sequencing to minimize the weighted completion time problem Q to the agent se- 
lection problem. This problem can be stated as follows: Given a set T of tasks, a partial 
order ^ on T, a length 1(f) G and a weight w(f) G for each task t G T, and a 
positive integer K, is there a one-processor schedule a for T that obeys the precedence 
constraints and for which the sum + ^(^)) ^ R 

Theorem 1. The agent selection problem is NP-hard. □ 

The rest of this paper consists of two parts. First, in Section 0] we describe an 
algorithm, Create-SSG, which automatically creates a feasible and evaluable SSG for 
a service condition x (assuming one exists). This algorithm extends and improves a 
“safety check” algorithm described in HI- Then, in Section 0 we develop algorithms 
to compute a feasible and evaluable SSG which is optimal (w.r.t. a cost function) as well 
as a heuristic algorithm which computes suboptimal solutions fast. 

4 Creating a Feasible and Evaluable SSG 

In this section, we develop an algorithm which takes a service condition x input 
and generates a feasible and evaluable SSG for x (assuming one exists). We use the 
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following definition to create edges in an SSG. This definition tells us that if a variable 
which appears as an argument in service atom a± is instantiated by another service atom 
02 , then oi is dependent on 02 and hence 02 may have to be executed before oi. 

Definition 9 (Dependent service conditions). A service condition Xj ^ said to be 
dependent on Xi if and only if the following holds: 

1. Case 1: Xi is a service atom of the form in{Xi, s{ti, . . . , tm))- 

a) If Xj is a service atom of the form in{Xj, , f„)) then 3j(l < j < n) 

s.t. root{tj) G root(Xi). 

b) If Xj is comparison atom of the form t\ = ^ 2 , then either ti is a variable 
and rootifi) S root{Xi) ort 2 is a variable and root{t 2 ) G root(Xi). 

c) If Xj is a comparison atom of the form t\ op t 2 , where op is one of <, <, >, >, 
then eitherti is a variable and ti G root{Xf) ort 2 is a variable and root(t 2 ) G 
root{Xi), or both. 

2. Case 2: Xi is a comparison atom of the form t\ op t 2 ^. 

a) If Xj is service atom of the form in{Xj, s(ti, . . . , tn)) then 3j(l < j < n) 
s.t. root(tj) G root{var{xi))- 

b) If Xj is o comparison atom of the form then either is a variable and 

root{tf) G root{var{xi)) or ti is a variable and root{ti) G root{var{xi))- 

c) If Xj is comparison atom of the form op ti, where op is one of <, <, > 
, >,then either t^ is a variable and root(tf) G root(var{xi)) orti is a variable 
androot{ti) G root{var{xi)) or both. 

The Create-SSG algorithm is given in Figure0 The Create-SSG algorithm creates 
an SSG where each vertex contains only one atom and each vertex is labeled with an 
agent that can execute the atom. The algorithm first computes the set of atoms {Ok) 
which do not depend on any other atoms. It keeps track of variables that are instantiated 
in the set V ar, and uses this set to determine which other atoms become evaluable. At 
each iteration of the while loop, the algorithm identifies the set of atoms that become 
evaluable, inserts the set of variables which are instantiated by those atoms, and creates 
the minimal number of edges implied by the dependency relations. 

Theorem 2. The Create-SSG algorithm generates a feasible and evaluable SSG, if one 
exists. □ 

We have implemented the Create-ssg algorithm on a Sun Ultral machine with 320 
MB memory running Solaris 2.6. We generated SDAs containing 10 services, each 
having 2 agents that provide that service. As the Create-ssg algorithm randomly picks 
one agent to label each vertex, we used this setting in all the experiments. We ran two sets 
of experiments. In the first set, we kept the number of dependencies constant and varied 
the number of conjuncts from 5 to 40. We repeated the same experiments when 10, 20, 
30, and 40 dependencies are present. For each combination of number of dependencies 
and conjuncts, we created 1000 service conditions and recorded the average running 
time. Figure Elshows the results. As seen from the figure, the execution time increases 
linearly with the number of conjuncts. The Create-SSG algorithm is extremely fast, 

if op is then at most one of ti or t 2 is a variable. 
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Create-SSG(x) 

/* Input: X : Xi ^ X 2 A . . . A Xn */ 

/* Output : an evaluable SSG (V,E) if one exists, NIL otherwise */ 

(I) ^':={xi.X2,--,Xn};(2) L':=0;(3) Var :=0; (4) £ := 0; 

(5) V :={xi I 1 < « < n}; 

(6) forall Xi 

(7) label Xi with s, s.t. s G SD[xi]-list 

(8) Ofc :={xi I Xi is either of the form in(X, s(args)) where args are ground, or of the form 

(9) ti = t 2 where at most one of ti or t 2 is a root variable, or of the form ti op t 2 , 

(10) where op €{<,>, <, >} and both ti and t 2 are constants }; 

(II) Var := Var U {root{Xi) \ {in{Xi, s{args)) G Ok or Xi = constant G Ok) 

(12) and Xi = root(Xi)}; 

(13) L-.= L-Ok- L'-=L'uOh, 

(14) while {L is not empty) do 

(15) ^ {xi I Xi G ^11 root variables in Xi are in Var}; 

(16) if card(iji') = 0 then Return NIL; 

(17) else Var := Var U {root{Xi) \ {in{Xi, s{args)) € 'P or 

(18) Xi = constant G P) and Xi = root(Xi)}; 

(19) forall pairs {Xi,Xj}’ Xj ^ s4. Xj is dependent on Xi £ L' 

( 20 ) E~Eu{{xi,Xi)h 

(21) L ■- L -E; L' ■- L' UE; 

(22) end(while) 

(23) Return (V, E); 

End-Algorithm 



Fig. 3. Create-SSG Algorithm 



taking less than 30 milliseconds for service conditions involving 40 conjunctions and 
25 dependencies. 

In the second set of experiments, we kept the number of conjuncts constant, and 
varied the number of dependencies from 10 to 50. We ran four experiments with 10, 20, 
30, and 40 conjuncts. Again, we generated 1000 service conditions for each combination 
and used the average running time. The results are given in FigureEl Again, the execution 
time increases linearly with the number of dependencies. 



5 Agent Selection Algorithms 

In this section, we describe two algorithms that take a service condition x as input and 
produce a sourced service graph as output. The first algorithm. A* -SSG, is an A* -based 
algorithm which finds an optimal solution. The A* -SSG algorithm maintains nodes n 
with the following fields: a (sub-)SSG, a cosf function value g{n) and a heuristic function 
value h{n). The algorithm starts out with an empty graph and builds evaluable and 
feasible sub-SSGs by considering one atom at a time. The algorithm expands one node 
at a time and maintains a list OPEN of nodes ordered in ascending order of g{n) 4- h{n) . 
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Fig. 4. Execution Time of Create-SSG (constant Fig- 5. Execution Time of Create-SSG (constant 
no of dependencies) no of conjuncts) 



At each stage, the A*-SSG algorithm expands the node with the smallest g{n) + h{n) 
value. In order to define the notion of an expansion, we first define a “semi-expansion.” 

Definition 10 (Semi-Expansion). G' = {V\ E') semi-expands G = (V, E) with atom 
Xi if 



1. Xi G Atoms{x) — Atoms{G), and 

2. G' is feasible and evaluable, and 

3. E C. E' , and 

4. either V' = V U {\i}, and \i i^ labeled with some agent a which provides the 
service referenced in x according to SDA. 

or V = {y — {u}) U {v'}, for some v € V such that v' = v A Xz. label(v) is 
an agent that provides the service referenced in x according to SDA. 



G' = {V , E') minimally semi-expands G with Xi iff 

1. G' expands G with Xi, and 

2. J3G"{y" , E") such that G" expands G with Xi and E" C E'. 

Finally, SemiExpansions{G) = {G' \ 3xi € {Atoms{x) — Atoms{G)) such 
that G' minimally semi-expands G with Xi}- 

A minimal expansion intuitively adds a new atom into G without removing any existing 
edges in E — however, some new edges might be added and/or some existing vertices 
might be expanded. 

Definition 11 (Expansion). Suppose G = (V, E) is as above and RR is some set of 
rewrite rules. G' = fV' , E') expands G = (V, E) with atom Xi ifG' is a semi-expansion 
w.r.t. Xi of some rewriting of the service condition GAtoms(G) Xj) using the rewrite 
rules in RR. 

We use Expansions{G) to denote the set of all expansions of G. 
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It is important to note that there may be many elements in Expansions{G) . We are now 
ready to dehne the cost function g{n) and the heuristic function h{n) associated with 
our A*-SSG algorithm. 

Definition 12 (Fnnctions g{n) and h{n)). Let G = (V, E) be the sub-SSG associated 
with a node of the A*-SSG algorithm, then g{n) and h(n) are defined as follows: 

g(n) = cost{V, E) 

h{n) = minc'={v' ,E')€Expansion{G){cost{V , E')) - cost{V, E) 

The heuristic function h gives the minimum cost increment when we include one more 
(service or comparison) atom in the current sub-SSG. 



A*-SSG(x, SD) 

/* Input: a service condition x *l 

/* Output: an evaluable SSG of x if one exists, NIL otherwise */ 

OPEN := 0 

Ground := {xi \ Xi is either of the form in(X, s(args)) where args are ground, or of 
the form ti = t2 where at most one of ti or t2 is a root variable, or of 
the form t\ op t2, (op € {<,>,<, >}) and both ti and t2 are constants }; 
forall Xi £ Ground do 
forall agents s that offer Xi 

create a sub-SSG G with one vertex v containing Xi nnd label s 
create a new node n with G 
insert n into OPEN 
sort OPEN in increasing order of /(n) 
while (OPEN / 0) ^ 
n := OPEN.head 
delete n from OPEN 
if n is a goal node then Retum(n.G) 
else /* expand and generate children */ 
insert all nodes n' € Expansion(n.G) into OPEN 
end(while) 

Return (NIL) 

End-Algorithm 



Fig. 6 . The A* -SSG Algorithm 



The A*-SSG algorithm is given in FigureEl To create evaluable sub-SSGs, the A*- 
SSG algorithm first computes the set of service atoms, Ground, which do not depend 
on any other service atoms, and hence are evaluable right away. It creates sub-SSGs 
containing just one atom from Ground. Moreover, the Expansions function only creates 
evaluable and feasible sub-SSGs. If the while loop of the algorithm terminates, this 
implies that no goal state was reached and hence there is no solution. A node n is a goal 
state for xAn.G is an evaluable and feasible SSG of x- Note that if the heuristic function 
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h used by the A* algorithm is admissible, then the algorithm is guaranteed to find the 
optimum solution (5]. The heuristic h is admissible iff h{n) < h*{n) where h*{n) is 
the lowest actual cost of getting to a goal node from node n. The following result shows 
that our heuristic is admissible. 

Theorem 3 (Admissibility of h). For all nodes n, h{n) < h*{n). Hence, the A*-SSG 
algorithm finds an optimal solution. □ 

5.1 Heuristic Algorithm 

In this section, we describe a greedy algorithm that uses heuristics to solve the agent 
selection problem. As the algorithm is a greedy one, it is not guaranteed to hnd an opti- 
mal solution. The algorithm starts out by creating a non-optimal feasible and evaluable 
initial solution. It iteratively improves the initial solution by applying a series of SSG 
transformations: Merge and Relabel that we introduce below. Merge merges two vertices 
of the SSG into one, whereas Relabel changes the label of a vertex. We first define these 
transformations, and then we describe our heuristic. 

Definition 13 (Merge). The Merge transformation takes an SSG G = {V,E) and two 
vertices x% t^nd \j cis input and generates another SSG G' = {V , E') as output such 
that 

1. If label(xi) = label(xj) <^tid either there exists an edge e G E between Xi tind Xj< 
or there exists no path between Xi tind Xj< then 

- V = {V - {xi, Xj}) U {Xi A Xil. and 

- E' = E- {(xhXk) I Xi e {Xj,Xi} or Xk £ {Xj,Xi}} U {(Xi A Xj,Xk)} U 

{{xi,Xi/\Xj)} 

2 . Otherwise, 

- V' = V andE' = E 

The Merge transformation merges two vertices into one by taking the conjunction of 
the service conditions in those two vertices. It does so only if the two vertices are labeled 
with the same agent. It deletes all edges that are between other vertices Xk and either 
Xi or Xj and inserts new edges that are now between that vertex and the merged vertex 
Xi A Xj - The following example illustrates how the Merge transformation is applied. 

Example 4 . Let xi = in{X, sc\{a)), X2 = in{Y, SC2{X)), X3 = sc^ib)) and 
X4 = in{W, sC4{Y, Z)). Suppose X = Xi A X2 A X3 A X4> and suppose we know that 
the following agents provide the services sci, . . . , SC4: 

sci : agentl,agent2 
SC2 : agent2,agentb 
SC3 : agents 
SC4 : agentl, agents 

Consider the feasible and evaluable SSG of Figure 0 If we apply the Merge trans- 
formation Merge{G, Xii X2). we get the SSG G' given in Figure|Hl We can apply the 
Merge transformation because xi and X2 are labeled with the same agent, agent 2 , and 
there exists an edge between them. 
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agent2 

in(X, sc 1(a)) 



agent2 
in(Y, sc2(X)) 



agents 
in(Z, sc3(b)) 



agent4 



in(W, sc4(Y,Z)) 



Fig. 7. SSG before Merge 




Fig. 8. SSG after Merge 



Proposition 1. IfG is evaluable and feasible, then so is Merge{G, Xii Xj)- 



Definition 14 (Relabel). The Relabel transformation takes an SSG G = {V, E) and a 
vertex \i as input and generates an SSG G' = (V' , E') as output such that 

1. V' = V and E' = E 

2 . label(xi) = Si € G, and label(xi) = S2 € G' for one Xi G ^ ottd Si,S2 G 
SD[xi]-list. 



Proposition 2. If G is evaluable and feasible, then so is Relabel{G , x)- 

The order in which these transformations are applied can be significant. For example, 
suppose we have two vertices v\ and V2 in an SSG. Suppose v\ is labeled with agent 
Ai and V2 is labeled with A2- Further, suppose a third agent A3 is able to process the 
service conditions in both v\ and V2- If we first apply the Relabel transformation to both 
vi and V2 and change both labels to A3, we can also apply the Merge transformation. 
In order to catch such possibilities, we need to consider all possible transformations 
in all possible orders. However, as the size of the SSG increases such a search space 
becomes too large to handle. Hence, instead of creating all possible SSGs, we propose 
a heuristic-based algorithm. 

The Greedy-Order algorithm is provided in Figure El It first creates a feasible and 
evaluable SSG by using the Create-SSG algorithm. It then creates a queue of vertices, 
sorted in decreasing order of cost. The algorithm examines one vertex at a time, and 
applies all possible transformation to this vertex. It chooses the best cost SSG among 
all the resulting SSGs, and sets this SSG as the current SSG. The algorithm terminates 
when it exhausts the vertices in the queue. 



6 Related Work 

To date, most existing work on what part of a service request should be assigned to 
which agent has been based on “matchmaking” efforts. In matchmaking efforts, agents 
advertise their services, and matchmakers match an agent requesting a service with one 
(or more) that provides it. Four of the best known examples of this class of work are 
given below. 
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Greedy-Order(x, SD) 

/* Input: a service condition x- 
/* Output: an evaluable SSG of x */ 

G{V, E) := Create-SSG(x); 

currSSG ~ G\ currGost := cost{V, E); 

create QUEUE and insert vertices of G into QUEUE; 

order vertices in QUEUE in decreasing order of cost; 

while (QUEUE 7 ^ emptyset) ^ 

Xi '■= QUEUE.dequeue; 

minSSG := currSSG; minCost := currGost; 
forall ixj s.t. G' = M.erge{currSSG, Xi, Xj) ^ G) do 

G'{V',E') ~ Merge{currSSG,Xi,Xj)\ 

if (cost{V' , E') < minGost) then 
minSSG := G'; minGost := cost{V' , E'); 
currSSG := minSSG; currGost := minGost; 
forall (s € SD\yi].list) do 
G'{V',E') ■- R.elahel{currSSG,Xi, s); 
if {costiy' , E') < minGost) then 
minSSG := G'; minGost := cost{V' , E'); 
currSSG := minSSG; currGost := minGost; 
end(while) 

Returnfcur r 5'S'G) ; 

End-Algorithm 



Fig. 9. The Greedy-Order Algorithm 



Kuokka and HaradaQ present the SHADE and COINS systems for matchmaking. 
SHADE uses logical rules to support matchmaking - the logic used is a subset of KIE and 
is very expressive. In contrast, COINS assumes that a message is a document (represented 
by a weighted term vector) and retrieves the “most similar” advertised services using 
the SMART algorithm of Salton flTlI . 

Decker, Sycara, and WilliamsonEl present matchmakers that store capability adver- 
tisements of different agents. They look for exact matches between requested services 
and retrieved services, and concentrate their efforts on architectures that support load 
balancing and protection of privacy of different agents. 

Arisha et. al. 0 develop matchmakers in which each agent advertises its services 
via an HTML style language. A metric space is defined on this language, and the job of 
matchmaking is now reduced to a nearest neighbor search on such a metric space. 

Vassalos and Papakonstantinou 111 511 develop a rule based matchmaking language. In 
this framework, when an agent sends a request for a service, the rules are used to match 
the requested service with an agent that provides it. 

In contrast to the above efforts, our work builds on top of a “matchmaker.” Match- 
makers may be used to implement the SDA described in our framework. However, we 
make novel use of the SDA. Specifically, we use costs to determine, of many possible 
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agents that could satisfy a service request, which ones should actually be assigned to the 
job. This determination not only takes into account possible methods to rewrite service 
requests, but also different possible assignments of agents to sub-requests. This yields 
two dimensions of complexity: rewriting of service conditions as well as assignment of 
agents to perform the task. 

7 Conclusions 

Agents are playing an increasingly important role in a vast variety of applications. 
These applications range from personalized marketing agents, personalized presenta- 
tion agents, agents to integrate heterogeneous databases and software packages, agents 
for supply chain management, agents for battlefield applications, and many, many others. 
This explosion of agent research has led to the development of several excellent agent 
development systems including, but not limited to systems such as Aglets from IBM Q, 
IMPACT III II . and Retsina III2I . 

We believe that for agents to be effective, they must be “small.” Small agents typically 
perform a small number of clearly articulated, well defined tasks (even though these tasks 
may Involve accessing a huge amount of data). Verifying a small program is much easier 
than a large program. In addition, small programs are easier to modify and maintain. 
Additionally, should these programs be mobile, then their requirements on a host machine 
are small (i.e. they have a small footprint on the host). By having a large number of small 
agents collaborate with one another, we may build large agent applications. 

In such situations, we will often have multiple agents that provide similar services. 
This situation is already widespread on the Internet. There are thousands of news sources, 
and hundreds of yellow page information sources. Sources on books and videos abound. 
When an agent wishes to access a service that it does not already provide, it must (i) find 
other agents that provide the desired services, and (ii) determine which of those agents 
to actually use to obtain the desired service. 

Problem (1) above has already been solved by “matchmaking” technology described 
in the preceding section. The goal of this paper is to address problem (ii). We believe that 
performance issues must drive the choice made in problem (ii). Thus, in this paper, we 
have proposed how an agent can make choices about how to execute a service condition 
and which agents should be selected to execute appropriate parts of (a perhaps rewritten) 
service condition. This problem, which we call the Agent Selection Problem {ASP for 
short) is proved in this paper to be NP-hard. We then develop two algorithms to solve 
this problem — an algorithm that computes an optimal solution, building on top of the 
well known A* algorithm, and a greedy algorithm. 
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Abstract. In this paper we show how to extend clausal temporal resolution to 
the ground eventuality fragment of monodic first-order temporal logic, which 
has recently been introduced by Hodkinson, Wolter and Zakharyaschev. While a 
finite Hilbert-like axiomatization of complete monodic first order temporal logic 
was developed by Wolter and Zakharyaschev, we propose a temporal resolution- 
based proof system which reduces the satisfiability problem for ground eventuality 
monodic first-order temporal formulae to the satisfiability problem for formulae 
of classical first-order logic. 



1 Introduction 



We consider the first-order temporal logic over the natural numbers TL{N) in a first- 
order temporal language T£. The language T C is constructed in the standard way 
(see i.e. lFis07lHWZUO|| ) from a classical (non-temporal) first-order language £ and a 
set of future-time temporal operators ‘0’ {sometime), ‘ O’ (always), ‘O’ (in the next 
moment), ‘ 14 "(until) and ‘ W ’ (unless, or weak until). Here, £ does not contain equality 
or functional symbols. 

Formulae in T£ are interpreted in first-order temporal structures of the form 971 = 
(D,I) , where D is a non-empty set, the domain of 971, and I is a function associating 
with every moment of time n S N an interpretation of predicate and constant symbols 
of £ over D. First-order (nontemporal) structures corresponding to each point of time 
n will be denoted by 97l„ = {D, /„) where /„ = I(n). Intuitively, the interpretations 
of T£-formulae are sequences of worlds such as 971 qj ■ ■ ■ , S9l„ .... An assignment 

in 17 is a function a from the set £v of individual variables of £ to 17. We require that 
(individual) variables and constants of T£ are rigid, that is neither assignments nor 
interpretations of constants depend on worlds. 

Ths truth-relation Tin '/’(or simply n (/j,if97l is understood) in the structure 
971 for the assignment a is defined inductively in usual way under the following semantics 
of temporal operators: 



n Op 

n (}p 

n □</? 

n |=“ pU ij: 

n \=^ pW Ip 



iff n-\- 1 p] 

iff there exists a m> n such that m p; 
iff m p for all m> n; 
iff there exists a m > n such that m ip and 
for every k G N, if n <k < m then k |=“ p : 
iff n |=“ pUip or n |=“ Op. 



F. Baader, G. Brewka, and T. Filer (Eds.): KI 2001, LNAI 2174, pp. 1 S-IT1 2001. 
(c) Springer- Verlag Berlin Heidelberg 2001 
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A formula ip is said to be satisfiable if there is a first-order structure 971 and an assignment 
a such that 97lo ‘/J. If 97lo for every structure 971 and for all assignments, then 
p is said to be valid. Note that formulae here are interpreted in the initial world 97io; that 
is an alternative but equivalent definition to the one used in IHWZOOI . 



2 Divided Separated Normal Form 



Our method works on temporal formulae transformed into a normal form. This normal 
form follows the spirit of Separated Normal Form (SNF) IFis911FDP(71|| and First-Order 
Separated Normal Form (SNF^) IFis9'2IFis97ll . However, we go even further. 

One of the main aims realized in SNF/SNF/ was inspired by Gabbay’s separation 
result IIGabS7ll . In accordance with this aim, formulae in SNF/SNFy comprise implica- 
tions with present-time formulae on the left-hand side and (present or) future formulae 
on the right-hand side. The transformation into the separated form is based upon the 
well-known renaming technique IIPG86I . which preserves satisfiability and admits the 
extension to temporal logic in (Renaming Theorems IFis97in . 

Another intention was to reduce most of the temporal operators to a core set. This 
concerns the removal of temporal operators represented as maximal fixpoints, i.e. O 
and W (Maximal Fixpoint Removal Theorems IIFis97ll 7. Note that the U operator can 
be represented as a combination of operators based upon maximal fixpoints and the 0 
operator (which is retained within SNF/SNF/). This transformation is based upon the 
simulation of fixpoints using QPTL [Wol82H . 

Now we add one additional aim, namely to divide the temporal part of a formula from 
its (classical) first-order part in such way that the temporal part is as simple as possible. 
The modified normal form is called Divided Separated Normal Form or DSNF for short. 
A Divided SNF problem is a triple <U,S^T > where S and U are the universal part 
and the initial part, respectively, given by finite sets of nontemporal first-order formulae 
(that is, without temporal operators), and T is the temporal part given by a finite set 
of temporal clauses. All formulae are written in £ extended by a set of predicate and 
propositional symbols. A temporal clause has one of the following forms: 



P{x) ^ O A Qi{^) (predicate step clause), 

i=l 



P^O A qj 
7=1 



(proposition step clause). 



P{x) OOQ(x) (predicate eventuality clause), 



p => OO? 



( proposition eventuality clause) 



where P,Q,Qi are predicate symbols, p,q,qj are propositional symbols, and => is 
a substitute for implication. Sometimes temporal clauses are called temporal rules to 
make distinctions between their left- and right-hand sides. Without loss of generality we 
suppose that there are no two different temporal step rules with the same left-hand sides 
and there are no two different eventuality rules with the same right-hand sides. An atom 
Q{x) or q from the right-hand side of an eventuality rule is called an eventuality atom. 
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We call examples of DSNF temporal problems. The semantics of a temporal problem 
is defined under the supposition that the universal and temporal parts are closed by the 
outermost prefixes DV, the initial part is closed only by universal quantifiers. In what 
follows we will not distinguish between a finite set of formulae X and the conjunction 
!\X of formulae in it. Thus the temporal formula corresponding to a temporal problem 

< W, 5, r > is ( DVW) A V5 A ( DVr). 

So, when we consider the satisfiability or the validity of a temporal problem we 
implicitly mean the corresponding formula, as above. 

Given the results about the renaming of subformulae and the removal of temporal 
operators mentioned above, we can state the general theorem about translation into 
DSNF as follows. 

Theorem 1. Any first-order temporal formula pin'TC can be translated into a temporal 
problem < > (i.e. DSNF of p) in a language TCI D TC extended by new 

propositional and predicate symbols such that ip is satisfiable if and only if <14,S,T > 
is satisfiable. 

For any formula ip its DSNF representation < U,S,T > can be constructed in poly- 
nomial time in the length of ip. (As a whole the transformation of p into < S,U,T > 
is similar to the familiar depth-reducing reductions of first-order formulae via the intro- 
duction of new names.) 

Example 1. Let us consider the following formula: p = (3x n~'Q(a:)) A {03yQ{y)). 
After transformation ptos. normal form, we get the following temporal problem: 

fsl.pil (tl.pi^Op2, \ 

\s2.p3j’ \f2. Pi(x)^0(P2(x)APi(a:))/’ 

7 / _ / Psix) D P 2 {x), u3. P 2 [x) D -^Q{x), u5. P 3 D BxP^ix) 

° \ m2. Psix) D Piix), m4. P 2 D 3yQiy), 

3 The Monodic Fragment and Merged Temporal Step Rules 

Following HHWZOOI we consider the set of all T £-formulae p such that any subformula 
of p of the form (}ip, T'fi, OV'j V'l ^ V'2, V'l ^ V '2 has at most one free variable. Such 
formulae are called monodic, and the set of monodic £-formulae is denoted by 71 £. In 
spite of its relative narrowness the monodic fragment provides a way for quite realistic 
applications. For example, temporal extensions of the spatial formalism RCC-8 IWolUUII 
lie within the monodic fragment. Another example is the verification of properties of 
relational transducers for electronic commerce lAVFYOOl which are expressed in the 
monodic language again. 

The decidability of TiC was proved in IHWZOOI while, in IWZOll , a finite Hilbert- 
style axiomatization of the monodic fragment of T L(N) has been constructed. However 
no deduction-based decision procedure for this class has yet been proposed. 

The notion ‘monodic’ is transferred from temporal formulae to temporal problems 
as follows. A problem < U,S,T > is monodic problem if all predicates occurring in 
its temporal part T are monadic. Every monodic formula is translated into DSNF given 
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by the monodic problem. In Example Dlboth the formula ip and its DSNF problem are 
monodic. 

The key role in propositional temporal resolution is played by so-called merged step 
clauses IFDPOl I . In the case of the monodic fragment, we can define an analogue of 
propositional merged step clauses and so formulate for monodic problems a calculus 
which is analogous to the propositional temporal resolution calculus (up to replacing the 
propositional merged clauses by the first-order merged clauses defined below) such that 
this calculus is complete for the so-called ground eventuality monodic fragment defined 
in the next section. 

Next we introduce the notions of colour schemes and constant distributions. Let 
V =<U,S,T >bea temporal problem. Let C be the set of constants occurring in V. 
Let T** = {Pi(x) => ORi{x), I 1 < i < K} and = {pj => Oxj \ I < j < k}he 
the sets of all predicate step rules and all propositional step rules of T, respectively. It 
is supposed that K > 0 and fc>0; ifiT = 0(fc = 0)it means that the set (T*’) 
is empty. (The expressions Ri{x) and Vj denote finite conjunctions of atoms /\ Qu{x) 

i 

and respectively.) 

i 

Let {Pi, . . .,Pk,Pk+i ■ ■ -,Pm}, 0 < iT < M, and {pi, . . . ,pk,Pk+i ■ ■ ■ ,Pm}, 
0 < k < m, he sets of all (monadic) predicate symbols and propositional sym- 
bols, respectively, occurring in T. Let Z\ be the set of all mappings from (1, . . . , M} 
to {0,1}, and 0 be the set of all mappings from {l,...,m| to {0,1}. An ele- 
ment 6 G A (9 G 0) is represented by the sequence [^(1), . . . ,S{M)] G {0, 1}^ 
([0(1), . . . , 6{m)] G {0, 1}™). Let us call elements of A and 0 predicate and proposi- 
tional colours, respectively. Let P be a subset of A, and 9 be an element of 0, and p be a 
map from C to P . A triple (P, 9, p) is called a colour scheme, and p is called a constant 
distribution. 



Note 1. The notion of the colour scheme came, of course, from the well known method 
of the decidability proof for the monadic class in classical first-order logic (see, for 
example, [BGG97I ). In our case we construct quotient structures based only on the 
predicates and propositions which occur in the temporal part of the problem, because 
only these symbols are really responsible for the satisfiability of temporal constraints. 
Besides, we have to consider so-called constant distributions, because unlike the classical 
case we cannot eliminate constants replacing them by existentially bounded variables - 
the monodicity property would be lost. 

For every colour scheme C = {P,9, p) let us construct the formulae Pc, Ac, Be in the 
following way. In the beginning for every y G P and for 9 introduce the conjunctions: 



A 


Pi{x) A A 


^Pi{x), 


Fe-- 


= /\ Pi ^ A ~^Pi, 


7(i) = l 


7(i)=0 






e{i)=l 9{i)=0 


A.y{x) 


= A 


Pi{x), 


Ae = 


A Pi, 




'y(^i) — lSci<K 








Bj{x) 


= A 


Ri{x), 


Bg = 


A A- 






9{i) — lSzi"^k 
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Now iFc, Ac, Be are of the following forms 

Tc= f\ 3xF^{x) hFe ^ f\ i^p(c) (c) A Vx \J F^{x), 

7GT cGC 7GT 

Ac = /\ 3xA,y{x)AAgA !\ Ap(c)(c)AVa; \J A,y{x), 

jer cgc 7gt 



Be = l\ 3xBj{x) A Bg A Bp(^c){c) AVx \J B~^{x). 

7G-T cGC 7GT 

We can consider the formula Tc as a ‘categorical’ formula specification of a quotient 
structure given hy a colour scheme. In turn, the formula Ac represents the part of 
this specification which is ‘responsible’ just for ‘transferring’ temporal requirements 
from the current world (quotient structure) to its immediate successors. The clause 
( IZ|V)(„4c ^ OBc) is then called a merged step rule. Note that if both sets {i \ i < 
iT, 7 G F,j{i) = 1} and {i \ i < k,9{i) = 1} are empty the rule {Ac => OBc) 
degenerates to (true => Otrue). 

Example 2. Let us return to the temporal problem obtained in the example Q The tem- 
poral part produces the following set of step merged clauses 

1. (pi A\/xPi{x)) ^ 0{P2 Ayx{P 2 {x) A Pi{x))), 

2. {pi A 3xPi{x)) ^ 0(f2 A 3x{P2{x) A Pi{x))), 

3. {VxPi{x)) => 0(Vx(P2(x) A Pi(x))), 

4. (3xPi(x)) ^ 0(3x(P2(x) a Pi(x))). 

For this problem iT = M = 2, k = m = 1. The problem does not contain any constants, 
and in this case the colour schemes are defined as pairs of the form {P,9). 

The first merged rule corresponds to the colour scheme ({[!,_,_]}, [!,_,_]) (the 
subformula 3xPi{x) A \/xPi{x) is reduced to \fxPi{x)). The second rule corresponds 
to ({[1, _], [0, _]}, [1, _]) (as usual the value of the empty conjunction /\ Pi{x) is 

ieil) 

true). The third and the fourth rules correspond to ({[1, _]}, [0, _]) and ({[1, _], 

[0, [0, respectively. 

The set of merged step rules for a problem <U,S,T > is denoted by mT. 

4 Resolution Procedure for Monodic Induction Free Problems 



A problem <U,S,T > is called induction free if T does not contain eventuality rules. 
In this section a derivation system based on a step resolution rule is given which is 
complete for the induction free monodic fragment. 

Definition 1 (step resolution rule). Let mT be the set of merged rules of a problem 
*7 lA , S , T {A 0)B) G vcip. Then the step resolution inference rule w.r.t. lA is the 
rule 



A^OB 

^A 



(O res ) 



with the side condition that the set lA U {B} is unsatisfiable.^ 

* The side condition provides the rule with the second (implicit) premise true => O giving 
this mle a usual resolution form. 
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Note 2. The test whether the side condition is satished does not involve temporal rea- 
soning and can be given to any first-order proof search procedure. 

By Step(W, mT) we denote the set of all formulae which are obtained by the step 
resolution rule w.r.t U from a merged clause A ^ OS in mT. Since mT is finite the 
set Step(T, mT) is also finite. 

Lemma 1 (soundness of step resolution). Let < U,S,T > be a temporal problem, 
and -'A G Step(U, mT). Then <U,S,T^ > is satisfiable if, and only if, <U U {-■.A}, 
5, T > is satisfiable. 

We describe a proof procedure for < U,S,T > by a binary relation > on (universal) 
sets of formulae, which we call a transition or derivation relation. In this section we 
define the derivation relations by the condition that each step Ui \> Ui+i consists of 
adding to the set Ui (to the state Ui) a formula from Step(Wi, mT). (In the next section 
this relation will be extended by a new sometime resolution rule). A finite sequence 
Ui) \> Ui \> U 2 \> ■ . ■ \> Un, where Uq = U, is called a (theorem proving) derivation for 

<u,s,t>.E 

Definition 2 (termination rule and fair derivation). A theorem proving derivation 
U = Uf)\>Ui\> . . .\> Un, n > 0,/or a problem <U,S,T > is successfully terminated 
if the set Un U S is unsatisfiable. The theorem proving derivation for a problem < U, 
S,T > is called fair if it either successfully terminates or, for any i > 0 and a formula 
-<A S StepiUi, mT), there is j > i such that -'A S Uj. 

Note 3. We intentionally do not include in our consideration the classical concept of 
redundancy (see IBGOll ) and deletion rules over sets of first-order formulae Ti because 
the main purpose of this paper is just new developments within temporal reasoning. 

As we can see only the universal part is modified during the derivation, the temporal and 
initial parts of the problem remain unchanged. 

Following [FDPOll we base our proof of completeness on a behavior graph for the 
problem < U,S,T >. Since, in this section, we are interested only in induction free 
problems we consider only so-called eventuality free behaviour graphs. 

Definition 3 (eventuality free behaviour graph). Given a problem V =< U,S,T > 
we construct a finite directed graph G as follows. Every node of G is a one-tuple (C) 
where C is a colour scheme for T such that the setU U Tc is satisfiable. 

For each node (C), C = (T, 0, p), we construct an edge in G to a node {C), C = 
(T', O' , p'), if U /\ Tc A Be is satisfiable. They are the only edges originating from (C). 
A node (C) is designated as an initial node ofG if S AU A Tq is satisfiable. 

The eventuality free behaviour graph H ofV is the full subgraph of G given by the 
set of nodes reachable from the initial nodes. 

^ In reality we can keep the states Ui in the form which is the most suitable for applying a first- 
order theorem prover procedure. For example, for a classical resolution-based procedure they 
could be saturated sets of clauses liBGOlll . 
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It is easy to see that there is the following relation between behaviour graphs of two 
temporal problems when one of them is obtained by extending the universal part of 
another one. 

Lemma 2. Let Vi =< Ui,S,T > and V 2 =< 1^2,5, T > be two TC problems such 
thatlLi C 1^2- Then the behaviour graph H 2 0 /P 2 ^ a subgraph of the behaviour graph 
Hi ofVi. 

Proof The graph H 2 is the full subgraph of Hi given by the set of nodes whose in- 
terpretations satisfy IA 2 and which are reachable from the initial nodes of Hi whose 
interpretations also satisfy ZY 2 . □ 

In the remainder of this section we will refer to an eventuality free behaviour graph 
simply as a behaviour graph. 

Definition 4 (suitable pairs). Let (C, C) where C = {F, 9, p), C = {F', 6' , p') be an 
(ordered) pair of colour schemes for H. An ordered pair of predicate colours ( 7 , 7 ^ 
where 7 G L, 7 ' € L' is called suitable if the formula U A F^i (x) A B^(x) is satisfiable. 
Similarly, the ordered pair of propositional colours {9,6') is suitable ifU A Fgi A Bg 
is satisfiable. The ordered pair of constant distributions {p, p') is called suitable if, for 
every c € C, the pair {p{c),p'{c)) is suitable. 

Lemma 3. Let H be the behaviour graph of a problem <U,S,F > with an edge from 
a node (C) to a node {C) of H, where C = {F, 9, p) and C = {F' , 9' , p'). Then 

— for every ■j G F there exists 7' G F' such that the pair (7, 7') is suitable; 

— for every 7' G F' there exists 7 G L such that the pair (7, 7') is suitable; 

— the pair of propositional colours {9, 9') is suitable; 

— the pair of constant distributions {p, p') is suitable. 

Proof To prove the first item it is enough to note that satisfiability of the expression 
U A Tc A Be implies satisfiability of W A (fix \J Fyix)) A 3xB^{x). This, in turn, 

7'GT' 

implies satisfiability of its logical consequence 14 A \/ 3x{F.yi (x) A B.y(x)). So, one 

y'er' 

of the members of this disjunction must be satisfiable. The second item follows from the 
satisfiability of the formula 14 A (Vx \/ B^{x)) A 3xF.yi{x). Other items are proved 

~f&r 

similarly. □ 

Let H be the behaviour graph of a problem <14,S,F > and H = (Co), . . . , (C„), . . . 
be a path in H where Ci = {Fi, 9i, pf. Let (/q = 5 U {!Fc,f} and (/„ = Fc^ A Bc^_^ 
for n > 1. From classical model theory, since the language C is countable and does not 
contain equality the following lemma holds. 

Lemma 4. Let nbe a cardinal, k > Hq. For every n > 0, if a setUyj{Qn\ is satisfiable, 
then there exists an L-model = {D, If) ofU U {Gn} such that for every 7 G Ln 
the set D{n,~i) = {a G D \ 9Jt„ ^ F.^(a)} is of cardinality k. 
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Definition 5 (run). By a run in II we mean a function from N ?o |J -Ti such that for 

iGN 

every n S N, r(n) S Fn and the pair {r(n), r(n + 1)) is suitable. 

It follows from the definition of H that for every c G C the function Vc defined hy 
rc(n) = Pn{c) is a run in II. 

Theorem 2. An induction free problem <11,5,1^ > is satisfiable if, and only if, there 
exists an infinite path II = (Cq), . . . , (C„), . . . through the behaviour graph H for 
< U,S,F > where (Cq) is an initial node of H. 



Proof (=>) Let 971= (Z9, 1) he a model of < (T, 5, T >. Let us define for every n S N 
the node (C), C = (Fn, 9n, Pn), as follows. 

For every a G Diet be a map from {1, . . . , M} to {0, 1}, and let be a map 
from M} to {0, 1} such that 




if Tin 1= Pi{a), 
if Tin P^{a) 




if Tin h 
if Tin P^ 



for every 1 < i < M. 

Now we define {"f(n,a) I a G D}, and p„(c) for every c G C. 

(Recall that, in accordance with our semantics, all constants are “rigid”, that is = 
for every u,v G N.) According to the construction {Fn, On, Pn) given above we 
can conclude thet the sequence (Cq), . . . , (C„), . . . where Cn = {Fn,0n, Pn), n G N, is 
a path through H. 

Proof (<;=) Following iHWZOOll take a cardinal k > Hq exceeding the cardinality of the 
set 5R of all runs in FI. Let us define a domain D = | r G 5R, ^ < k}. Then for 

every n G N and for every S G A 



II {(r,^) G D I r(n) 




ifSGFn, 

otherwise. 



So, for every n G N it follows that D = |J D(^n,j) where = {(f, C) ^ 

D I r{n) = 7}. Hence by Lemma ^ for every n G N there exists an £-structure 
Tin = {D, In) which satisfies U U {Gn}- Moreover, we can suppose that c^" = (rc, 0) 
andD(„.^) = {(r,^) G D \ Tin |= C))} forevery 7 G r)i. A first-order temporal 

model that we sought is 9H = {D,I) where I(n) = /„ for all n G N. To be convinced of 
that let us show validity of an arbitrary step rule O {Pfix) => QRi{x)) in 9J1. Namely, 
let us show that, for every n > 0 and for every (r, G D, if \= Pi{{r,^)), then 
Tln+i H Suppose r{n) = ^ G Fn and r(n + 1) = 7' G Fn+i, that is 

(r,^) G D(^n,-f) and (r,^) G If 97l„ |= Pi{{r,0) then y{i) = 1. It follows 

that Ri{x) is embedded in Bj{x) ( if we consider Ri{x) and Fy{x) as sets). Since 
the pair (7, 7') is suitable it follows that the conjunction Rfix) is embedded in F^ix). 
Together with ^ F.y’i^{T, ^)) this implies that 



Corollary 1 (completeness of the step resolution). If an induction free problem V = 
< IA,S,F > is unsatisfiable, then every fair theorem proving derivation for < U,S, 
T > succesfully terminates. 
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Proof Let U = Wq l> . . . I> > . . . [> be a fair theorem proving derivation for 
a problem V = < U,S,T >■ The proof proceeds by induction on the number of 
nodes in the behaviour graph H of V, which is finite. If H is empty then the set W U 5 is 
unsatisfiable. In this case the derivation is successfully terminated because the set U5 
includes U U S and therefore it is unsatisfiable too. 

Now suppose H is not empty. Since < U,S,T > is unsatisfiable Theorem |2 tells 
us that all paths through H starting from initial nodes are finite. Let (C) be a node of H 
which has no successors. In this case the set W U {Be} is unsatisfiable. Indeed, suppose 
U U {Be} is satisfiable, and {D', /') is a model of W U {Be}- Then following the proof 
of the previous theorem we can define a colour scheme C such fhat {D' , /') ^ Tc'- 
Since Be A Te' is satisfiable fhere is an edge from fhe node (C) fo fhe node C' in fhe 
confradiction with the choice of (C) as having no successor. Since the derivation is fair, 
there is a step when -'Ae is included to a state Ui D U. This implies removing the 
node (C) from the behaviour graph Hi of the problem < Ui^S^T > because the set 
{Te,^Ae} is not satisfiable. By lemmaQil follows thaf Hi is a proper subgraph of H. 

Now we can apply induction hypothesis to the problem <Ui,S,T > and to the fair 
derivation Ui\> . . .\>Un- □ 

Example 3. Let us return to ExampleO We can apply step resolution (w.r.t. Hq) to the 
second clause because the set Uq U {p 2 A \/x{P 2 {x) A Pi (a;))} is unsatisfiable: 

(pi AVa;Pi(a:)) ^ 0(P2 AVa;(P 2 (a:) APi(a;))) 



5 Resolution Procedure for Ground Eventuality Monodic 
Problem s 

A problem < U,S,T > is called a ground eventuality problem if T confains only 
propositional eventuality rules. In this section a derivation system based on the step 
resolution rule defined above and on a new sometime resolution rule defined below is 
given which is complete for the ground eventuality monodic fragment. 

Definition 6 (sometime resolntion rule). Let mT be the set of merged rules of a prob- 
lem <U,S,T ^ — r (f) B~^ j . . . — r (f) Bji y IS a subset of m*/ , and p — r C ) ^ tj is 

a propositional eventuality rule in T. Then the sometime resolution inference rule w.r.t. 
U is the rule 



Ai ^ OBi, . . . ,An ^ OBn P^O()q 

n 

-■( V A) V -ip 



where the following (loop) side condition has to be satisfied 



n 



U U {Bm} I — 'q/\ \J Ai 
1=1 



for all 1 < m < n. 
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Under the side condition given above V Ai can be considered as an invariant formula 

i—1 

that provides the derivability of D O “’<7 from U U T- Again, as in the case of step 
resolution, the test of whether the side conditions are satisfied does not involve temporal 
reasoning and can be given to any first-order proof search procedure. 

By Res(ZT, T) we denote the set of all formulae which are obtained by the sometime 
resolution rule w.r.t W from a set of merged clauses Ai => QBi, . . . An => O in mT 
and an eventuality clause p ^ O 0<7 in T. Since T and mT are finite the set Step(fT, T) 
is also finite (up to renaming bound variables). The sometime resolution rule is sound 
in the sense similar to the soundness of the step resolution rule (see Lemma QJ. 

To take into account eventuality clauses we modify the notion of the behaviour graph 
given in the previous section by introducing an additional (eventuality) component to 
every node. 

Definition 7 (ground eventuality behaviour graph). 

Given a problem V =< IA,S,T > we construct a finite directed graph G as follows. 
Every node of G is a two-tuple (C, E) where 

— C is a colour scheme for T such that the set lA A Tc is satisfiable; 

— E is a subset of eventuality atoms occurring in T It will be called the eventuality 
set of the node {C, E). 

For each node (C, E), C = (E, 9, p), we construct an edge in G to a node {C), C = 
{r' , 9' , p'), if U A Tc A Be is satisfiable and E' = E^ U E^ where 

= {q \ Q ^ E and Eg/ \f g}, 

= {q I there exists an eventuality rule {p => OO?) G T such that 
Eg \~ p and Eg' \f q}. 

They are the only edges originating from (C,E).A node (C, 0) is designated as an initial 
node of G if S AU A Tc is satisfiable. The eventuality free behaviour graph H of P is 
the full subgraph of G given by the set of nodes reachable from the initial nodes. 

Let H be the behaviour graph of a problem V, n, n' be nodes of a graph H. We denote 
the relation “n' is an immediate successor of n ” by n — ?► n', and the relation “n' is a 
successor of n ” by n — n'. 

A node n of iT is called step inference node if it has no successors. A node n' of 
H is called sometime inference node if it is not a step inference node and there is an 
eventuality atom qinP such that for every successor n' = {€', E'), q G E' holds. 

Lemma 5 (existence of a model). 

Let V be a problem, H be the behaviour graph ofP such that the set of initial nodes 
of H is not empty and the following condition is satisfied: 

VnV(73n'(n — n' A g ^ T') (1) 

where n, n' are nodes of H, n = (C,E), n' = (C',E'), and q belongs to the set of 
eventuality atoms ofV. Then V has a model. 
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Proof We can construct a model for V as follows. Let no be an initial node of H 
and , . . . , 5 m be all eventuality atoms of V. Let 7T be a path no , • . . , ni , . . . , Wm , ■ ■ ■ , 
n,„+i, . . . , n 2 m, . . where Ukm+j = {Ckm+j,Ekm+j) is a successor of Ukm+j-i in H 
such that Qj ^ Ekm+j (for every fc > 0, 1 < j < to). 

Let us take the sequence (Co), . . . , (Ci), . . . , (C^), . . . , (Cm+i), ■ • ■ , (C 2 m), ... in- 
duced by n. Now let us consider this sequence as an infinite path in the eventuality free 
behaviour graph for the induction free problem < W, S^* > where T* is obtained 
from T by removing all (propositional) eventuality rulesjjThen the first-order temporal 
model dJfl — {D,I) constructed by the theorem^lfor < U, S, T* > from the sequence 
(Co), . . . , (Cl), . . . , {Cm), ■ • ■ , {Cm+i), • ■ • , (C 2 m), ■ • ■ is a model for P =< U,S,T >■ 
Indeed, all nontemporal clauses and all step clauses of V are satisfied on this structure 
immediately by the dehnition of <14, S,T* >■ Let us take an arbitrary eventuality clause 
Pj => O ()Qj of T, a moment of time I G N and the l-th element (C, E) on U. If Fc Pj 
then Pj 0()qj is satished at the moment I, i.e. dJti ^ {pj D OO^y)- If Fc ^ Pj 
we take a node xikm+j which is a successor of (C, E) on II. By the construction of 
n it follows that qj ^ Ekm+j- We conclude that there exists a successor {C ,E') of 
(C, E) along the path to rikm+j such that Fc h qj, otherwise Ij G Ekm+j would hold. 
It implies that pj => O is satished at the moment I as well. □ 

To provide the completeness of the sometime resolution rule for the problems which 
contain more than one eventuality atom such problems have to be augmented in the 
following way. 

Definition 8 (augmented problem). Let us introduce for every eventuality atom q oc- 
curring in T a new propositional symbol Wq. An augmented proplem is a 

triple < 5, > where 

l^aug =uvJ{wqZ:) {py q)\{p^ OOq) G T}, 

j-aug I (p ^ g j-y 

The necessity for the augmentation even in the propositional case was shown in IDFOOl . 
It is obvious that the augmentation is invariant with respect to satishabilityQ 

Now we extend the notion of the derivation relation introduced in the previous section 
as follows: each step I4i t> Ui^i consists of the adding to the set Ui a formula from 
Step(Wi,mT) or from Res(Z/fi,T). Correspondingly, the notion of the (fair) theorem 
proving derivation is modihed. 

Theorem 3 (completeness of the step+sometime resolution). If a ground eventuality 
problem V =< IA,S,T > is unsatisfiable, then every fair theorem proving derivation 
for =< iS, > is succesfully terminated. 



^ To retain the set of propositional symbols of T* the same as of T we can add to T* degenerates 
step rules of the form p =+ O true. 

Both the augmentation and including degenerates rules (see the previous footnote) can result 
in the violation of the condition that there are no different step rules with the same left-hand 
sides. However this violation is eliminated easy. 
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Proof The proof proceeds by induction on the number of nodes in the behaviour graph 
H of ^ which is finite. The cases when H is empty graph or there exists a node n in 
H which has no successors are considered in the same way as in the proof of Corollary[I] 
Now we consider another possibility when H is not empty and every node in H has 
a successor. It is enough to prove that there is a formula ip G Res(ZT, T) such that for 
some node (C, E) of H the formula 14 A Ec /\ ip is, unsatisfiable. 

In this case because V is unsatisfiable fhe following condition (the negation of the 
condition O) of the existence of a model given in Lemma|3) holds: 

3n3gVn'(n — n' D q G E') (2) 



where n, n' are nodes of iT, n = {I,E), n' = {I'^E'), and q belongs to the set of 
eventuality atoms of V. 

Let Uq = (Co,Eq) be the node defined by the first existential quantifier of the 
condition 0 . Let qo be the eventuality atom defined by the second existential quantifier 
of the condition Q. Letp => OO^o be the eventuality rule containing go (on the right). 

Let 3 be a finite nonempty set of indexes, {n^ | i G 3} be the set of all successors 
of Uq. (It is possible, of course, that 0 G 3.) Let , . . . be the set of all immediate 

successors of uq, = (Cy , Ei.) for 1 < J < fc. To simplify denotations in this proof 
we will represent merged rules Aci => O^Ci (Aci . => O^Ci . ) simply as Ai ^ OEi 
(A^^ ^ OEi-), and formulae Eci (.T^c^p simply as {Ei-'). 

Consider two cases depending on whether the merged rule Ao 0<Bo (or any of 
=> OSi, i G 3 ) is degenerated or not. 

1. Let Ao = Bo = true . It implies, that U h -■go. Indeed, since go G Ei^ for all 
l< 3 <k then Ei^ 1/ go in accordance with the definition of the ground eventuality 
behaviour graph. Again similar to the proof of the CorollaryQJ suppose that WU {go} 
is satisfiable, and (73', I') is a model of 77 U {go}- Then we can construct a colour 
scheme C such that (73', I') p Ec and therefore Ec L go. Since , . . . ni^ is 
the set of all immediate successors of no and Bq = true it holds that there exists j, 
l<3 <k, such that Cij = C'. We conclude that go ^ Ey because of Ec L go. 

It contradicts the choice of the node no- So, 77 I 'go, and the following sometime 

resolution inference is realized 

true=^Otrue p=>00<7o 



The behaviour graph for the problem <77U{-'p},5,T >isa proper subgraph of 77 . 
Indeed, if Fg^ h p then no has to be removed from El. If Fg^ p then a predecessor 
(C, 73), C = {r, 6, p), of the node no such that Eg \- p has to be removed from 77. 
The set of such predecessors is not empty because the eventuality set of every initial 
node of 77 is empty. 

The same argument holds if one of Ai ^ O Bi, i & '3 , is degenerate. 

2. Let neither ,Ao ^ QBq nor any Ai ^ QBi,i G 3, are degenerate. We are going 
to prove now that in this case the sometime resolution rule 



{Ai ^ OB^ I i G {0} U 3} p^ OOqo 

( A V -•p 

ie{o}u3 



(Ores) 



is applied. We have to check the side conditions for the sometime resolution rule. 
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- By arguments similar those given in item 1 we conclude that the sets 

{go}) for all i G {0} U 3 are unsatisfiahle. It implies that W U {Bi} I <qo for 

all i e {0} U 3. 

- Let us show that U U {Bi} h V Aj for alH € {0} U 3. Consider the case 

ie{o}ua 

i = 0, for other indexes arguments are the same. Suppose that QA U {Bq} U 
{ f\ -'Aj}) is satisfied in a structure (!?',/'). Let C he a colour scheme 

i<j<fe 

of {D',I'), that is |= Tc- Then there is a node = {Ci.,Ei.), 

I < J < k, which is an immediate successor of Uq, such that Ci^ = C , 
and hence {D' , I') \= Ai -. However it contradicts the choice of the structure 
{D',I'). 

After applying the (Ores) rule given above we add to 1/ its conclusion, which is 
equivalent to the set of formulae {-•Ai V -•p \ i G {OjUT}. To prove that the 
behaviour graph of the extended problem will contain less nodes than H we have 
to consider two cases depending on whether go G Eg or not. 

a) Let us suppose go ^ Eg. Then Eg \~ p because qg G Ei^ and there is an adge 
from (Co, Eg) to (Ci^ , Liii ). In this case the node no has to be removed from H. 
Recall that Eg h .4o by the definition of . 

b) Let us suppose qg G Eg. Since the eventuality set of every initial node is 

empty there exists a predecessor Ug of Hq and a path n(, = (Cg, Eq) . . . , = 

(C'm^E!^) from n'o to no, m > 1, C'^ = (EO,,0'^, p'^), such that n'„ = no, 
Fg>^ h p, and qg G E} for all 1 < j < m. The last condition implies 

Fe'. I — '90 for all 1 < j < rn. (3) 

That is just the place where we have to involve in our arguments the augmenting 
pair for p => O Ogo- Let this pair be presented by the following clauses 

P^Owg ( 4 ) 

Wg^qgAp GW'^F ( 5 ) 

From the clause @ it follows that Eg'^ h wg. From the clause © and the 
condition ©) it follows that for all 1 < j < m it holds Fg> h p. It implies that 
Fg^ h p, in particular, since — no. So, Ecg A -'p is unsatisfiahle. Therefore 
Uo has to be removed from the behaviour graph after extending by the 
formula -i^o V ~'p (the same as every node n^, / G T is removed after including 
the formula -lAi V -ip. 

Now all possible cases related to the properties of E[ have been considered. □ 

Lemma 6 (existence of a model, one eventuality case). Let V be a problem such that 
V contains the only eventuality atom qg. Let H be the behaviour graph ofV such that 
the set of initial nodes of H is not empty and the following condition is satisfied: 

Vn(go ^ E D 3n'(n n' A go ^ E')) (6) 

where n, n' are nodes of L[, n = (C, E), n' = (C', E'). Then V has a model. 
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Proof We use model construction of the proof of Lemma|3 taking m = 1. □ 

Corollary 2 (completeness of the one eventuality case). If a ground eventuality prob- 
lem V —< UjSj'T > is unsatisfiable, and T contains at most one eventuality atom then 
every fair theorem proving derivation for V =< U, Sj'T > is succesfully terminated. 

Proof This corollary is obtained by analysing the proof of Theorem 01 given above. 
Firstly, using Lemma El and supposing go to be the only eventuality atom of T we can 
strengthen the condition (0 to the following 3n(go ^ E A Vn'(n — n' go G E')) 
where n, n' are nodes of the eventuality graph H for the problem V, n = (C,E), 
n' = {C\E'). This immediately implies that the case 2(b) of the previous proof, where 
augmentation has been required, is excluded from the consideration. □ 



6 Conclusion 

It has been known for a long time that first-order temporal logic over the natural numbers 
is incomplete ISzaSbll . that is there exists no finitary inference system which is sound 
and complete for the logic, or equivalently, the set of valid formulae of the logic is not 
recursively enumerable. The monodic fragment is the only known today fragment of 
first-order temporal logic among not only decidable but even recursively enumerable 
fragments which has a transparent syntactical definition and a finite inference system. 

The method developed in this paper covers a special subsclass of the monodic frag- 
ment, namely the subclass of the ground eventuality monodic problems. Nevertheless 
this subclass is still interesting w.r.t. both its theoretical properties and possible area of 
applications. The first statement is confirmed in parficular by the fact that if we slightly 
extend its boundaries admitting a binary relation in the step rules then its recursive enu- 
merability will be lost. The second is justified in particular by the observation that the 
temporal specifications for verifying properties of transducers considered in | |Spi00| | are 
proved to be not simply monodic but monodic ground eventuality problems. 

One of the essential advantages of the method given above follows from the com- 
plete separation of the classical first-order component. As a result classical first-order 
resolution can be applied as a basic tool in the temporal proof search (to solve side and 
termination conditions, which are expressed in classical first-order logic). That imme- 
diately gains access to all benefits, both theoretical and practical, of resolution based 
decision procedures IFLH’TUl I . because the first-order formulae produced by temporal 
rules are very simple and they cannot change the decidability/undecidability of the initial 
fragment. Future work includes extending these results to wider fragments of first-order 
temporal logic, and implementing this approach. 

It might also be interesting to decompose the present separated and ‘global’ temporal 
inferences into a mix of resolution-like ‘local’ rules. That will involve revision of the 
resolution method without skolemization for classical logic developed in IZamSTII . 

We thank anonymous referees for their helpful comments and suggestions. Unfor- 
tunately our possibilities to fully respond to them are limited by space restrictions. This 
work was supported by EPSRC under research grants GR/M46631 and GR/N081 17. 
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Abstract. Gomputing the most specific concept (msc) is an inference 
task that allows to abstract from individuals defined in description logic 
(DL) knowledge bases. For DLs that allow for existential restrictions or 
number restrictions, however, the msc need not exist unless one allows for 
cyclic concepts interpreted with the greatest fixed-point semantics. Since 
such concepts cannot be handled by current DL-systems, we propose to 
approximate the msc. We show that for the DL ACE, which has concept 
conjunction, a restricted form of negation, existential restrictions, and 
value restrictions as constructors, approximations of the msc always exist 
and can effectively be computed. 



1 Introduction 

The most specific concept (msc) of an individual 5 is a concept description 
that has b as instance and is the least concept description (w.r.t. subsump- 
tion) with this property. Roughly speaking, the msc is the concept description 
that, among all concept descriptions of a given DL, represents b best. Closely 
related to the msc is the least common subsumer (Ics), which, given concept 
descriptions C'i,...,C'„, is the least concept description (w.r.t. subsumption) 
subsuming Ci, . . . ,C„. Thus, where the msc generalizes an individual, the Ics 
generalizes a set of concept descriptions. 

In 12111141 . the msc (first introduced in ^^) and the Ics (first introduced in 
0) have been proposed to support the bottom-up construction of a knowledge 
base. The motivation comes from an application in chemical process engineering 
m, where the process engineers construct the knowledge base (which consists of 
descriptions of standard building blocks of process models) as follows: First, they 
introduce several “typical” examples of a standard building block as individuals, 
and then they generalize (the descriptions of) these individuals into a concept 

* This work was carried out while the author was still at the LuFG Theoretische 
Informatik, RWTH Aachen, Germany. 
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description that (i) has all the individuals as instances, and (ii) is the most 
specific description satisfying property (i). The task of computing a concept 
description satisfying (i) and (ii) can be split into two subtasks: computing the 
msc of a single individual, and computing the Ics of a given finite number of 
concepts. The resulting concept description is then presented to the knowledge 
engineer, who can trim the description according to his needs before adding it 
to the knowledge base. 

The Ics has been thoroughly investigated for (sublanguages of) Classic |dl2l 
II 211 1| . for DLs allowing for existential restrictions like ACE 0, and most recently, 
for ACEAf, a DL allowing for both existential and number restrictions |T2 . For all 
these DLs, except for Classic in case attributes are interpreted as total functions 
it has turned out that the Ics always exists and that it can effectively be 
computed. Prototypical implementations show that the Ics algorithms behave 
quite well in practice m 

For the msc, the situation is not that rosy. For DLs allowing for number 
restrictions or existential restrictions, the msc does not exist in general. Hence, 
the first step in the bottom-up construction, namely computing the msc, can- 
not be performed. In [3, it has been shown that for ACM, a sublanguage of 
Classic, the existence of the msc can be guaranteed if one allows for cyclic 
concept descriptions, i.e., concepts with cyclic definitions, interpreted by the 
greatest fixed-point semantics. Most likely, such concept descriptions would also 
guarantee the existence of the msc in DLs with existential restrictions. However, 
current DL-systems, like FaCT m and RACE jO], do not support this kind of 
cyclic concept descriptions: although they allow for cyclic definitions of concepts, 
these systems do not employ the greatest fixed-point semantics, but descrip- 
tive semantics. Consequently, cyclic concept descriptions returned by algorithms 
computing the msc cannot be processed by these systems. 

In this paper, we therefore propose to approximate the msc. Roughly speak- 
ing, for some given non-negative integer k, the k- approximation of the msc of 
an individual b is the least concept description (w.r.t. subsumption) among all 
concept descriptions with b as instance and role depth at most k. That is, the 
set of potential most specific concepts is restricted to the set of concept descrip- 
tions with role depth bounded by k. For (sublanguages of) ACE we show that 
fc-approximations always exist and that they can effectively be computed. Thus, 
when replacing “msc” by “fc-approximation” , the first step of the bottom-up 
construction can always be carried out. Although the original outcome of this 
step is only approximated, this might in fact suffice as a first suggestion to the 
knowledge engineer. 

While for full ACE our fc-approximation algorithm is of questionable practi- 
cal use (since it employs a simple enumeration argument), we propose improved 
algorithms for the sublanguages EC and EC^ of ACE. (EC allows for conjunction 
and existential restrictions, and EC^ additionally allows for a restricted form of 
negation.) Our approach for computing fc-approximations in these sublanguages 
is based on representing concept descriptions by certain trees and ABoxes by 
certain (systems of) graphs, and then characterizing instance relationships by 
homomorphisms from trees into graphs. The fc-approximation operation then 
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Table 1. Syntax and semantics of concept descriptions. 



Construct name 


Syntax 


Semantics 


EC 


EC^ 


ACE 


top-concept 


T 


A 


X 


X 


X 


bottom-concept 


T 


0 




X 


X 


conjunction 


cnD 


cFrilP 


X 


X 


X 


existential restriction 


3r.C 


{x G A \ 3y: (x,y) G P A y G C'} 


X 


X 


X 


primitive negation 


-nP 


A\W 




X 


X 


value restriction 


Vr.C 


{x G A \ Vy : (x,y) G P ^ y G C^} 






X 



consists in unraveling the graphs into trees and translating them back into con- 
cept descriptions. In case the unraveling yields finite trees, the corresponding 
concept descriptions are “exact” most specific concepts, showing that in this 
case the msc exists. Otherwise, pruning the infinite trees on level k yields k- 
approximations of the most specific concepts. 

The outline of the paper is as follows. In the next section, we introduce the 
basic notions and formally define fc-approximations. To get started, in Section El 
we present the characterization of instance relationships in £C and show how 
this can be employed to compute fc-approximations or (if it exists) the msc. In 
the subsequent section we extend the results to £C^, and finally deal with ACE 
in Section El The paper concludes with some remarks on future work. Due to 
space limitations, we refer to PI for all technical details and complete proofs. 

2 Preliminaries 

Concept descriptions are inductively defined with the help of a set of construc- 
tors, starting with disjoint sets Nc of concept names and Nn of role names. In 
this work, we consider concept descriptions built from the constructors shown 
in Table [D where r G Nfi denotes a role name, P G Nc a concept name, and 
C,D concept descriptions. The concept descriptions in the DLs EC, EC^, and 
ACE are built using certain subsets of these constructors, as shown in the last 
three columns of Table 0 

An ABox ^ is a finite set of assertions of the form (a, h) : r (role assertions) or 
a : C (eoncept assertions) , where a, b are individuals from a set Nj (disjoint from 
Nq U Nfi), r is a role name, and C is a concept description. An ABox is called 
£-ABox if all concept descriptions occurring in A are £-concept descriptions. 

The semantics of a concept description is defined in terms of an interpre- 
tation X = (A,N), The domain Z\ of I is a non-empty set of objects and the 
interpretation function A maps each concept name P G Nc to a set P^ C A, 
each role name r G Nfi to a binary relation r^ C Ax A, and each individual 
b G Nj to an element aA G A such that a ^ b implies yf b^ (unique name 
assumption) . The extension of A to arbitrary concept descriptions is inductively 
defined, as shown in the third column of Table Q An interpretation I is a model 
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of an ABox A iff it satisfies (a^, lA) G for all role assertions (a, b) : r G A, 
and G for all concept assertions a ■. C G A. 

The most important traditional inference services provided by DL-systems 
are computing the subsumption hierarchy and instance relationships. The con- 
cept description C is subsumed by the concept description D (C Q D) iff 
C for all interpretations I; C and D are equivalent (C = D) iff they 
subsume each other. An individual a G Nj is an instance of C w.r.t. A (a Ga C) 
iff a?' G for all models X of A. 

In this paper, we are interested in the computation of most specific con- 
cepts and their approximation via concept descriptions of limited depth. The 
depth depth (C) of a concept description C is defined as the maximum of nested 
quantifiers in C. We also need to introduce least common subsumers formally. 

Definition 1 (msc, fc-approximation, Ics). Let A be an C-ABox, a an indi- 
vidual in A, C, Cl, . . . , C-concept descriptions, and fc G N. Then, 

— C is the most specific concept (msc) of a w.r.t. A fmsc_ 4 (a)^ iff o. Ga C, 
and for all C-concept descriptions C , a Ga C' implies C C C ; 

— C is the /c-approximation of a w.r.t. A iff a Ga C, depth(C) < 

k, and for all C-concept descriptions C , a Ga C' and depth(C') < k imply 
C C C'; 

— C is the least common subsumer (Ics) o/Ci,...,C„ (1cs(Ci, . . . , C„)^ ijf 
Ci C C for all i = 1, . . . ,n, and for all C-concept descriptions C , Ci C C 
for all i = 1, . . . , n implies C Q C' . 

Note that by definition, most specific concepts, ^-approximations, and least com- 
mon subsumers are uniquely determined up to equivalence (if they exist) . 

The following example shows that in DLs allowing for existential restrictions 
the msc of an ABox-individual b need not exist. 

Example 1. Let C be one of the DLs £C, or ACC. Consider the £-ABox 
A — {{b,b) : r}. It is easy to see that, for each n > 0, 6 is an instance of the 
£-concept description 

Cn := 3r.---3r .T. 

n times 

The msc of b can be written as the infinite conjunction n„>o C„, which, however, 
cannot be expressed by a (finite) /1-concept description. As we will see, the fc- 
approximation of b is the £-concept description rio<n<fc Cn. 



3 Most Specific Concepts in £C 

In the following subsection, we introduce the characterization of instance rela- 
tionships in ££, which yields the basis for the algorithm computing fc-approxi- 
mations (Section 13.211 . All results presented in this section are rather straightfor- 
ward. However, they prepare the ground for the more involved technical problems 
one encounters for ££-,. 
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3.1 Characterizing Instance in SC 

In order to characterize instance relationships, we need to introduce descrip- 
tion graphs (representing ABoxes) and description trees (representing concept 
descriptions). For an SC-description graph is a labeled graph of the form 
Q = (V,E,£) whose edges vrw G E are labeled with role names r G fV/j and 
whose nodes v G V are labeled with sets £{v) of concept names from Nc- The 
empty label corresponds to the top-concept. An SC-description tree is of the 
form Q — {V,E,vq,£), where (V,E,£) is an £C-description graph which is a tree 
with root G V. 

SC-concept descriptions can be turned into SC-description trees and vice 
versa P|: Every £C-concept description C can be written (modulo equivalence) 
as C = Pi n . . . n n 3ri.C\ n . . . n 3rm-Cm with Pi G Nc U {T}. Such a 
concept description is recursively translated into an SC-description tree G(C) = 
(V,E,vo,£) as follows: The set of all concept names Pi occurring on the top- 
level of C yields the label £{vq) of the root Vq, and each existential restriction 
Bri-Ci yields an ri-successor that is the root of the tree corresponding to Q. For 
example, the concept description 

C := Bs.(Q n 3r.T) n Br.(Q n 3s. T) 

yields the description tree depicted on the left hand side of Figure [D 

Every SC-description tree Q = (V,E,vo,£) is translated into an SC-concept 
description Cg as follows: the concept names occurring in the label of vq yield 
the concept names in the top-level conjunction of Cg, and each vorv G E yields 
an existential restriction 3r.C7, where C is the SC-concept description obtained 
by translating the subtree of G with root v. For a leaf v G V, the empty label is 
translated into the top-concept. 

Adapting the translation of SC-concept descriptions into SC-description trees, 
an SC- ABox A is translated into an SC-description graph G{A) as follows: Let 
Ind(A) denote the set of all individuals occurring in A. For each a G Ind(A), 
let Ca ■■= Ha :D^A D, if there exists a concept assertion a : D G A, otherwise, 
Ca := T. Let G{Ca) = (Va, Ea,a,£a) denote the SC-description tree obtained 
from Ca- (Note that the individual a is defined to be the root of G(Ca); in 
particular, a G Va-) W.l.o.g. let the sets Va, a G Ind(A) be pairwise disjoint. 
Then, G{A) := (V,E,£) is defined by 

^ ■ Uaelnd(^) 

- E--= {arb I (a, 6) : r € A} U UaGind(^) and 

- £{v) := £a{v) for v G K- 

As an example, consider the SC- ABox 

A = {a : P r\ 3s. (Q FI 3r.P □ 3s. T), b : PF\ Q,c : 3r.P, 

{a,b) ■- r, (a,c) : r,(b,c) : s}. 

The corresponding SC-description graph G(A) is depicted on the right hand side 
of Figure □ 
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a(c) 



i>0 : 0 



vi ■■ {Q} «3 : {Q} 



1>2 : 0 1)4 : 0 



a(^) 




b-.{P,Q} 



c : 0 



Fig. 1. The £C-description tree of C and the £C-description graph of A. 



Now, an instance relationship, a C, in £C can be characterized in terms 
of a homomorphism from the description tree of C into the description graph 
of A', a homomorphism from an £C-description tree Q = (V,E,vq,£) into an 
^^-description graph H = {VhtEh,(.h) is a mapping ip : V — Vh such that 
(1) £(v) C for all v G V, and (2) (p(v)r(p(w) G Eh for all vrw G E. 

Theorem 1. EF Let A be an £C-ABox, a G lnd(.A) an individual in A, and 
C an £C-concept deseription. Let G{A) denote the £L- description graph of A 
and G(C) the £L-description tree of C. Then, a Ga C iff there exists a homo- 
morphism ip from G{C) into G(A) such that ip{vo) = a, where vq is the root of 

G{C). 

In our example, a is an instance of C, since mapping vq on a, Vi on Wi, i = 1, 2, 
and U 3 on b and V 4 on c yields a homomorphism from G(C) into G(A). 

Theorem [D is a special case of the characterization of subsumption between 
simple conceptual graphs jOj , and of the characterization of containment of con- 
junctive queries p. In these more general settings, testing for the existence of 
homomorphisms is an NP-complete problem. In the restricted case of testing ho- 
momorphisms mapping trees into graphs, the problem is polynomial Thus, 
as corollary of Theorem D we obtain the following complexity result. 

Corollary 1. The instance problem for £L can be decided in polynomial time. 

Theorem ^ generalizes the following characterization of subsumption in £L in- 
troduced in P). This characterization uses homomorphisms between description 
trees, which are defined just as homomorphisms from description trees into de- 
scription graphs, but where we additionally require to map roots onto roots. 



Theorem 2. m Let C, D be £C-concept descriptions, and let G(C) and G{D) 
be the corresponding description trees. Then, C Q D iff there exists a homomor- 
phism from G{D) into G{C). 

3.2 Computing fe- Approximations in £C 

In the sequel, we assume A to be an £C-ABox, a an individual occurring in 
A, and k a non-negative integer. Roughly, our algorithm computing mscfe^^(a) 
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works as follows: First, the description graph G{A) is unraveled into a tree with 
root a. This tree, denoted T(a, G{A)), has a finite branching factor, but possibly 
infinitely long paths. Pruning all paths to length k yields an SC-description 
tree Tk{a,G{A)) of depth < k. Using Theorem [□ and TheoremQ one can show 
that the i£C-concept description Cji(^a,G(A)) is equivalent to mscfc_^(a). As an 
immediate consequence, in case T{a,G{A)) is finite, yields the msc of 

a. In what follows, we define T(a,G{A)) and 7fe(a, t/(Al)), and prove correctness 
of our algorithm. 

First, we need some notations: For an £C-description graph G = {V,E,£), 
p = voriVir 2 ■ ■ ■ TnVn is a path from vq to Vn of length \p\ = n and with label 
ri ■ ■ ■ r„, if Vi-iTiVi S E for all 1 < i < n. The empty path (n = 0) is allowed, 
in which case the label of p is the empty word e. The node is an ri • • • r„- 
successor of vq ■ Every node is an £-successor of itself. A node v is reachable from 
Vq, if there exists a path from vg to v. The graph G is cyclic, if there exists a 
non-empty path from a node in G to itself. 

Definition 2. Let G = (y,E,£) and a € V. The tree T{a,G) of a w.r.t. A is 
defined by E{a,G) '■= , E'^ , a,£'^) with 

— := {p \ p is a path from a to some node in G}, 

— := {prq \ p,q G and q = prw for some r G Nji and w G V}, 

— £'^(p) := £{v) if p is a path to v. 

For k G the tree Tk{a,G) of a w.r.t. G and k is defined by Tk{a,G) '■= 
with 

-V^:= {pGV^\\p\<k}, 

— El := E^ n {VH xNrX V;i), and 

— ^Hp) ■= ^^(P) for P&Vl'. 

Now, we can show the main theorem of this section. 

Theorem 3. Let A be an £C-ABox, a G Ind(A), and fc G N. Then, Cj-,^(a,Q{A)) 
is the k- approximation of a w.r.t. A. If, starting from a, no cyclic path in A can 
be reached (i.e., T{a,G{A)) is finite), then Cp(a,G(A)) Ihe msc of a w.r.t. A; 
otherwise no msc exists. 

Proof sketch. Obviously, there exists a homomorphism from Ek{a,G{A)), a tree 
isomorphic to G{C'jy^a,g(A}))j into G(A) with a mapped on a. By Theorem [0 
this implies a Cr^,{a,g{A))- 

Let C be an £C-concept description with a Ga C and depth(C) < k. Theo- 
remnimplies that there exists a homomorphism ip from G(C) into G{A). Given (p, 
it is easy to construct a homomorphism from G(C) into 7fc(a, G{A)). Thus, with 
Theorem|2l we conclude C-ji(^a,g(A)) E E. Altogether, this shows that Cji,(^a,g{A)) 
is a fc-approximation as claimed. 

Now, assume that, starting from a, a cycle can be reached in A, that is, 
T{a,G{A)) is infinite. Then, we have a decreasing chain Co □ Ci Z1 •••of 
/c-approximations Ck (= CTk{a,g{A))) with increasing depth k, k > 0. From 
Theorem |21 we conclude that there does not exist an £C-concept description 
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subsumed by all of these fc-approximations (since such a concept description 
only has a fixed and finite depth). Thus, a cannot have an msc. 

Conversely, if 7~(a, Q(A)) is finite, say with depth k, from the observation 
that all fc'-approximations, for k' > k, are equivalent, it immediately follows 
that C-j-(^a,g{A)) is the msc of a. □ 

Obviously, there exists a deterministic algorithm computing the fc-approximation 
(i.e., Cj-i^(^a,g(A))) in time 0{\A\^). The size \A\ of A is defined by 

1^1 := |lnd(^)| + |{(a, &) : r | (a, 5) : r e A}\ + E ici, 

a-.C^A 

where the size \C\ of C is defined as the sum of the number of occurrences of 
concept names, role names, and constructors in C. Similarly, one obtains an 
exponential complexity upper bound for computing the msc (if it exists) . 

Corollary 2. For an £C-ABox A, an individual a G Ind(^), and k G fi, the k- 
approximation of a w.r.t. A always exists and can be computed in time 0{\A\^). 

The msc of a exists iff starting from a no cycle can be reached in A. The 
existence of the msc can be decided in polynomial time, and if the msc exists, it 
can be computed in time exponential in the size of A. 

In the remainder of this section, we prove that the exponential upper bounds 
are tight. To this end, we show examples demonstrating that /c-approximations 
and the msc may grow exponentially. 

Example 2. Let A = {{a,a) : r, (a, a) : s}. The SC-description graph G(A) as well 
as the £C-description trees 7i(a, G(A)) and 72(o, G(A)) are depicted in Figure |3 
It is easy to see that, for A: > I, 7k(a,Q(A)) yields a full binary tree of depth k 
where 

— each node is labeled with the empty set, and 

— each node except the leaves has one r- and one s-successor. 

By Theorem El ^Tk{a,g(A)) is the fc-approximation of a w.r.t. A. The size of 
^Tk(a,g(A)) is Moreover, it is not hard to see that there does not exist an 
££-concept description (7 which is equivalent to but smaller than CTk{a,g(A))- 

The following example illustrates that, if it exists, also the msc can be of expo- 
nential size. 

Example 3. For n > 1, define An '■= {(«i,ai-i-i) : r, (oi,ai+i) : s | 1 < i < n}. 
Obviously, An is acyclic, and the size of An is linear in n. By Theorem El 
C-j-(ai,An) is the msc of oi w.r.t. An- It is easy to see that, for each n, T(oi, M„) 
coincides with the tree 77(oj G{A)) obtained in ExampleO As before we obtain 
that 

“ G'T(ai,g{A)) is of size exponential in |M„|; and 

— there does not exist an £C-concept description C equivalent to but smaller 
than Cr(ai,g{A))- 
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g{A) : 



Ti(a.eM)) : 

a : 0 



o 




ara : 0 asa : 0 



a : 0 



T2(a,g(^)) : 



a : 0 




ara : 0 



asa : 0 





arara : 0 arasa : 0 asara : 0 asasa : 0 



Fig. 2. The f£-description graph and the £C-description trees from Example 0 

Summarizing, we obtain the following lower bounds. 

Proposition 1. Let A be an SC-ABox, a G Ind(Al), and fc G N. 

— The size o/msc_ 4 jc(a) may grow with 

— If it exists, the size o/msc^(a) may grow exponentially in |.A|. 

4 Most Specific Concepts in £C^ 

Our goal is to obtain a characterization of the (fc-approximation of the) msc in 
£C^ analogously to the one given in Theorem 0 for £C. To achieve this goal, first 
the notions of description graph and of description tree are extended from £L to 
£C^ by allowing for subsets of NcU{-'P \ P G A^c}U{T} as node labels. Just as 
for £C, there exists a 1-1 correspondence between £C-,-concept descriptions and 
££-, -description trees, and an £C-,-ABox A is translated into an £C-,-description 
graph G{A) as described for £C-ABoxes. The notion of a homomorphism also 
remains unchanged for , and the characterization of subsumption extends to 
£C^ by just considering inconsistent £C-,-concept descriptions as a special case: 
C C £) iff C = T or there exists a homomorphism ip from Q{D) into Q{C). 

Second, we have to cope with inconsistent £’£-,-ABoxes as a special case: for 
an inconsistent ABox A, a C is valid for all concept descriptions C, and 
hence, msc^(a) = T. However, extending Theorem Q with this special case does 
not yield a sound and complete characterization of instance relationships for 
£C^. If this was the case, we would get that the instance problem for £C^ is 
in P, in contradiction to complexity results shown in uni, which imply that the 
instance problem for ££-, is coNP-hard. 

The following example is an abstract version of an example given in Cni; it 
illustrates incompleteness of a naive extension of Theorem Q from £C to £C^ . 

Example 4- Consider the £C^-concept description C = Pr\ 3r.{P □ 3r.~'P) and 



r}; G{A) and G{C) are depicted in Figure 0 Obviously, there does not exist 
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Wo : {P} 

r 

Wi : {P} 



i"2 : {-'P} 



Fig. 3. The £C-,-description graph and the £C^-description tree from ExampleQl 



a homomorphism (p from Q(C) into G{A) with ip{wo) = a, because neither 
P G ^(^ 2 ) nor -iP G ^( 62 ). For each model I of A, however, either G or 
G and in fact, G C^. Thus, a is an instance of C w.r.t. A though 

there does not exist a homomorphism tp from G{C) into G(A) with ‘p(wo) = a. 

In the following section, we give a sound and complete characterization of in- 
stance relationships in £C^ , which again yields the basis for the characterization 
of fc-approximations given in Section 14.21 

4.1 Characterizing Instance in £C^ 

The reason for the problem illustrated in Example 0 is that, in general, for the 
individuals in the ABox it is not always fixed whether they are instances of 
a given concept name or not. Thus, in order to obtain a sound and complete 
characterization analogous to Theorem 0 instead of G{A), one has to consider 
all so-called atomic completions of G{A). 

Definition 3 (Atomic completion). Let G — (V,E,£) be an £C^-description 
graph and Nq := {P G Nc \ exists v G V with P G £{v) or ->P G £{v)}. An 
£C^-description graph G* = (F,P,P) is an atomic completion of G if, for all 

V gV, 

1- £{.v) C £*{v), 

2. for all eoneept names P G Nf, either P G £*(v) or ->P G £*(v). 

Note that by definition, all labels of nodes in completions do not contain a con- 
flict, i.e., the nodes are not labeled with a concept name and its negation. In 
particular, if G has a conflicting node, then G does not have a completion. It is 
easy to see that an £’£_,-ABox A is inconsistent iff G(A) contains a conflicting 
node. For this reason, in the following characterization of the instance relation- 
ship, we do not need to distinguish between consistent and inconsistent ABoxes. 

Theorem 4. m Let A he an £C^-ABox, G{A) = (V,E,£) the eorrespond- 
ing description graph, C an £C^-concept description, G(C) = {Vc,Ec,wq,£c) 
the corresponding description tree, and a G Ind(A). Then, a G^ C iff for each 
atomic completion G{A)* ofG(A), there exists a homomorphism ip from G{C) 
into G{A)* with ip(wo) = a. 
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The problem of deciding whether there exists an atomic completion Q(A)* such 
that there exists no homomorphism from G{C) into G{A)* is in coNP. Adding 
the coNP-hardness result obtained from m, this shows 

Corollary 3. The instance problem for £C^ is coNP- complete. 

4.2 Computing fe- Approximations in 

Not surprisingly, the algorithm computing the fc-approximation/msc in £C does 
not yield the desired result for £C^. For instance, in Example 0, we would get 
Cr{a,g{A)) = Pn3r.3r.(-'P)n3r.(Pn3r.3r.(-'P)). But as we will see, msc_ 4 (a) = 
P n 3r.(P n 3r.-iP) □ 3r.(P □ 3r.3r.-iP), i.e., msc_ 4 (a) C C'j-(a,A)- 

As in the extension of the characterization of instance relationships from 
£C to £C-,, we have to take into account all atomic completions instead of the 
single description graph G{A). Intuitively, one has to compute the least concept 
description for which there exists a homomorphism into each atomic completion 
of G{A). In fact, this can be done by applying the Ics operation on the set of all 
concept descriptions g{A)“) obtained from the atomic completions G{A)* 
oiG{A). 

Theorem 5. Let A be an £C^-ABox, a G Ind(A), and fc G N. If A is inconsis- 
tent, then mscfc_^(a) = msc^(a) = _L. Otherwise, let {G{A)^, . . . ,G{A)^} be the 
set of all atomic completions ofG{A). 

Then, ^cs{Cr,,(a,g{A)0^ ■ ■ ■ ^^Tk(a,g{Ar)) = mscfe_^(a). If starting from a, no 
cycle can be reached in A, then \cs{C-i-i^a,g(AP)i ■ ■ ■ i^T{a,g{A)'^)) = rnsc^(a); 
otherwise the msc does not exist. 

Proof sketch. Let A be a consistent ££_,-ABox and . . . , the atomic 

completions of G{A). By definition of Cx,.(a,g{Ay)i there exists a homomorphism 
TTi from CTf,{a,g{AY) i’^to G{AY for all 1 < z < n. Let Ck denote the Ics of 
{^Tk(a,g{Ay)^ ■ ■ ■ ^^Tk(a,g{A)^)}- The characterization of subsumption for £C^ 
yields homomorphisms (pi from G{Ck) into G{Cx,,{a,g{AY)) for all 1 < z < n. 
Now it is easy to see that TTiOp^ yields a homomorphism from G{Ck) into G(Ay, 
1 < z < n, each mapping the root of G{Ck) onto a. Hence, a G^ Ck. 

Assume C with depth(C") < k and a Ga C' . By Theorem 0 there exist 
homomorphisms ipi from G{C) into G{AY for all 1 < z < n, each mapping the 
root of G{C) onto a. Since depth(C") < k, these homomorphisms immediately 
yield homomorphisms ip'^ from G{C) into G{Cj-^(^a,g{AY)) all 1 < z < n. Now 
the characterization of subsumption yields C'Tfc(o,e(^)-) E C' for all 1 < z < rz, 
and hence Ck E C . Thus, Ck = mscfc^^(a). 

Analogously, in case starting from a, no cycle can be reached in A, we con- 
clude lcs(C 7 -(a . . . , C-j-(^a,g{A)‘^_)) = msc_ 4 (a). Otherwise, with the same ar- 

gument as in the proof of Theorem y] it follows that the msc does not exist. □ 

In Example 0 we obtain two atomic completions, namely G(AY with £^( 62 ) = 
{P}, and G{AY with C‘{b 2 ) = {“'T’}. Now Theorem 0 implies msc_ 4 (a) = 
\c.s{Cr{a,g{AY)^^T{a,g{AY)), which is equivalent to 

P n 3r.(P n 3r.^P) n 3r.{P n 3r.3r.^P). 
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The examples showing the exponential blow-up of the size of A:-approx- 
imations and most specific concepts in Ed can easily be adapted to ££-, . However, 
we only have a double exponential upper bound (though we strongly conjecture 
that the size can again be bounded single-exponentially): the size of each tree 
(and the corresponding concept descriptions) obtained from an atomic comple- 
tion is at most exponential, and the size of the Ics of a sequence of £C-,-concept 
descriptions can grow exponentially in the size of the input descriptions |2j ■ 
Moreover, by an algorithm computing the Ics of the concept descriptions 
obtained from the atomic completions, the /c-approximation (the msc) can be 
computed in double exponential time. 

Corollary 4. Let A be an EC^-ABox, a € Ind(M), and k € IN. 

— The k- approximation of a always exists. It may be of size \A\^ and ean be 
computed in double- exponential time. 

— The msc of a exists iff A is inconsistent, or starting from a, no cycle can be 
reached in A. If the msc exists, its size may grow exponentially in \A\, and 
it can be computed in double- exponential time. The existence of the msc can 
be decided in polynomial time. 

5 Most Specific Concepts in ACS 

As already mentioned in the introduction, the characterization of instance rela- 
tionships could not yet be extended from E£^ to ACE. Since these structural char- 
acterizations were crucial for the algorithms computing the (fc-approximation of 
the) msc in EC and EC^, no similar algorithms for ACE can be presented here. 
However, we show that 

1 . given that Nc and are finite sets, the mscfe ^ 4 ( 0 ) always exists and can 
effectively be computed (cf. Theorem |^; 

2. the characterization of instance relationships in EC is also sound for ACE 
(cf. Lemma U, which allows for approximating the ^-approximation; and 

3. we illustrate the main problems encountered in the structural characteriza- 
tion of instance relationships in ACE (cf. Example EJ . 

The first result is achieved by a rather generic argument. Given that the signa- 
ture, i.e., the sets Nq and Nr, are fixed and finite, it is easy to see that also 
the set of AL^-concept descriptions of depth < k built using only names from 
Nc U Nr is finite (up to equivalence) and can effectively be computed. Since the 
instance problem for ACE is known to be decidable m, enumerating this set 
and retrieving the least concept description which has a as instance, obviously 
yields an algorithm computing msCfc_^(a). 



Theorem 6. Let Nq and Nr be fixed and finite, and let A be an ACE- AB ox built 
over a set Nj of individuals and Nc U Nr. Then, for L G IN and a G Ind(M), the 
k- approximation of a w.r.t. A always exists and can effectively be computed. 
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Note that the above argument cannot be adapted to prove the existence 
of the msc for acyclic ./U2r-ABoxes unless the size of the msc can be bounded 
appropriately. Finding such a bound remains an open problem. 

The algorithm sketched above is obviously not applicable in real applications. 
Thus, in the remainder of this section, we focus on extending the improved 
algorithms obtained for SC and £C^ to ACS. 



5.1 Approximating the fc- Approximation in ACS 

We first have to extend the notions of description graph and of description 
tree from £C^ to ACS'. In order to cope with value restrictions occurring in 
A/l^-concept descriptions, we allow for two types of edges, namely those labeled 
with role names r S Npi (representing existential restrictions of the form 3r.C) 
and those labeled with Vr (representing value restrictions of the form Mr.C). 
Again, there is a 1-1 correspondence between AfT-concept descriptions and ACS- 
description trees, and an AGP-ABox A is translated into an AGP-description 
graph G{A) just as described for £"£-ABoxes. The notion of a homomorphism also 
extends to ACS in a natural way. A homomorphism Lp from an AGP-description 
tree % = {Vh,Eh,vq,£h) into an AC£-description graph G = (V,E,£) is a 
mapping tp : Vh — > V satisfying the conditions (1) and (2) on homomorphisms 
between ££-description trees and ££-description graphs, and additionally (3) 
(p{v)\/r(p{w) € E for all vVrw € Eh. 

We are now ready to formalize soundness of the characterization of instance 
relationships for ACS. 

Lemma 1. EF Let A be an ACS-ABox with G(A) = {V, E,i), C an ACS-concept 
description with Gc = {^CtEc,vq,£c)j o.'nd a € Ind(A). 

If there exists a homomorphism (p from Gc into G(A) with <p(vo) = a, then 
a C. 

As an immediate consequence of this lemma, we get a ^Tk{a,g(A)) for all 
k > 0, where the trees T{a,G{A)) and Tk{a,G{A)) are defined just as for SC. 
This in turn yields msCfc_^(a) C Eji,(a,g{A)) 3 .nd hence, an algorithm computing 
an approximation of the fc-approximation for ACS. In fact, such approximations 
already turned out to be quite usable in our process engineering application P|. 

The following example now shows that the characterization is not complete 
for ACS, and that, in general, Cji(^a,g{A)) ^ rnsc/c_^(a). In particular, it demon- 
strates the difficulties one encounters in the presence of value restrictions. 

Example 5. Consider the ACS-KBox. 

A:= {a-. P,bi : P □ Vs.P □ 3r.P, 62 : PH 3r.(PH 3s : P), 

(a, 61 ) : r, ( 0 , 62 ) : r, ( 61 , 62 ) : r}, 

and the ACS-concepi description C = 3r.(Vs.Pn 3r.3s.T); G{A) and G{C) are 
depicted in Figure El Note that G(A) is the unique atomic completion of itself 
(w.r.t. Nc = {P}). 
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Fig. 4. The ,4C£^-description graph and the ,4C2^-description tree from Example 0 



It is easy to see that there does not exist a homomorphism from G{C) into 
G{A) with (f{wo) = a. However, a £_a C: For each model I of A, either does 
not have an s-successor, or at least one s-successor. In the first case, S Vs.P, 
and hence yields the desired r-successor of in (Vs.P □ 3r.3s.T)^. In the 
second case, it is S (3s. T)^, and hence yields the desired r-successor of 
a?-. Thus, for each model I of A, oA £ . 

Moreover, for /c = 4, Cxi{a,A) is given by P 13 3r.(P 3 Vs.P 3 3r.P 3 3r.(P 3 
3r.(P 3 3s. P))) 3 3r.(P 3 3r.(P 3 3s. P)). It is easy to see that % C- 

Hence, Cr^(a,A) n C C Cr^(a,A)i which implies msc 4 ,^(a) 3 Cr^(a,A)- 

Intuitively, the above example suggests that, in the definition of atomic com- 
pletions, one should take into account not only (negated) concept names but 
also more complex concept descriptions. However, it is not clear whether an ap- 
propriate set of such concept descriptions can be obtained just from the ABox 
and how these concept descriptions need to be integrated in the completion in 
order to obtain a sound and complete structural characterization of instance 
relationships in ACE. 

6 Conclusion 

Starting with the formal definition of the /c-approximation of msc we showed 
that, for ACE and a finite signature Nn), the fc-approximation of the msc 
of an individual b always exists and can effectively be computed. For the sublan- 
guages EC and EC^, we gave sound and complete characterizations of instance 
relationships that lead to practical algorithms. As a by-product, we obtained 
a characterization of the existence of the msc in EC- / EC^- ABoxes, and showed 
that the msc can effectively be computed in case it exists. 

First experiments with manually computed approximations of the msc in 
the process engineering application were quite encouraging used as inputs 
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for the Ics operation, i.e., the second step in the bottom-up construction of the 
knowledge base, they led to descriptions of building blocks the engineers could 
use to refine their knowledge base. In next steps, the run-time behavior and the 
quality of the output of the algorithms presented here is to be evaluated by a 
prototype implementation in the process engineering application. 
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Abstract. In this paper I want to argue that the combination of evolu- 
tionary algorithms and neural networks can be fruitful in several ways. 
When estimating a functional relationship on the basis of empirical data 
we face three basic problems. Firstly, we have to deal with noisy and 
finite-sized data sets which is usually done be regularization techniques, 
for example Bayesian learning. Secondly, for many applications we need 
to encode the problem by features and have to decide which and how 
many of them to use. Bearing in mind the empty space phenomenon, it 
is often an advantage to select few features and estimate a non-linear 
function in a low-dimensional space. Thirdly, if we have trained several 
networks, we are left with the problem of model selection. These prob- 
lems can be tackled by integrating several stochastic methods into an 
evolutionary search algorithm. The search can be designed such that it 
explores the parameter space to find regions corresponding to networks 
with a high posterior probability of being a model for the process, that 
generated the data. The benefits of the approach are demonstrated on a 
regression and a classification problem. 



1 Learning Based on Empirical Data 

If one wants to learn a functional relationship from empirical data, then the 
goal of network training is to recognize a structure in the data or an underlying 
process and to generalize this knowledge to former unknown data points. Bishop 
characterizes the goal of network training as building a statistical model of the 
process which generates the data [Bishop, 1995l|Hipley., 199^|Hertz et al., IMTl . 
If we approximate the data-generating process the error over all former unseen 
patterns will be minimal. 

1.1 Regularization 

A sound discussion on a theoretic level of the generalization problem can be 
found in [Vapnik, 19821 [Vapnik, 1995|| . Vapnik states that the problem of density 
estimation based on empirical data is ill posed, i.e., small changes in the learning 
situation can result in a totally different model; for example little distortions in 
the target data. The theory of regularization shows that instead of minimizing 
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the difference between the target data and the output of the model, a regularized 
error function 

E = Eo + XEfi (1) 

should be minimized where Ejj is the error on the data and A is a weighting 
factor. Efi is an additional term that measures the complexity of the model; 
for example the often used weight decay regularizer in neural network train- 
ing jKrogh fc Hertz, 1992HBishop, 199^ . Thus, regularization is not an optional 
possibility, but a fundamental technique |Ramsay fc Silverman, 1997j . One cru- 
cial problem is to determine the weighting factor A. In case of neural networks, 
its optimal value depends on the size of the network, i.e., the number of weights, 
the weight initialization, as well as the patterns used for training, and the noise 
in the data. Often this value is determined by cross-validation which is clearly 
suboptimal for the reasons just given. Furthermore, an additional data set is nec- 
essary to determine this parameter experimentally in a series of training runs; 
i.e., additional computational effort has to be investigated to fit the parameter A, 
while on the other hand less data points are left for the actual training process. 

Adjusting this value properly has been solved for neural networks by using 
a Bayesian learning algorithm. It was introduced by MacKay [MacKay, I992a| 
and provides an elegant theory to prevent neural networks from overfitting by 
determining A during the training process without the necessity of additional val- 
idation data. Still different models will be found depending on the initialization 
and the network topology (Fig.EJi), which can then be compared on basis of their 
posterior probability. A short review of Bayesian learning for neural networks 
as used in this work is given in section |2| (see [MacKay, 1992a| [Bishop, 1995| or 
|[Gutjahr, lyyytjlTagg, 2000] for a detailed discussion on Bayesian learning). 

1.2 Curse of Dimensionality 

In practical applications only a limited amount of data is available for deter- 
mining the parameters of a model. The dimensionality of the input vector must 
be in a sensible relation to the number of data points. By adding more features 
the information content of the input about the target increases, but at the same 
time the number of data points per dimension decreases in order to determine 
the parameters. That means that the class of functions in which the solution 
is searched increases greatly with every additional component. This problem 
is called the curse of dimensionality |Whitej_J^^ [Bisho£^_J^9^ or the empty 
space phenomenon [Scott &: Thompsor^^^^sTISiTvemmi^^^ . A consequence 
is that renunciation of supplementary information leads to a more precise ap- 
proximation of the underlying process in a low-dimensional input space. Tabled 
relates the dimensionality of the input vector to the number of patterns that is 
necessary to approximate a simple non-linear function [Silverman, 198d| . Note 
that for most of the known benchmark problems in machine learning databases, 
these values are not reached. 

A further aspect which is to be considered is the relation between the number 
of input features used and the size of the regularization term. Adding features 
results in a higher value of the regularization term, since the network structure 
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Table 1. Number of patterns that are necessary to estimate a multivariate normal 
distribution in the origin with a mean square error below 0.1 



dimensions 


number of patterns 




dimensions 


number of patterns 


1 


4 




6 


2.790 


2 


19 




7 


10.700 


3 


67 




8 


43.700 


4 


223 




9 


187.000 


5 


768 




10 


842.000 



grows. That is, if the additional features contribute only little information, they 
nevertheless lead to a stronger punishment of the complexity. In practical exper- 
iment one observes that learning in high dimensional input spaces often leads to 
almost linear solutions (cf. Fig0] see also |Kagg, 21)1701 ). 

1.3 Model Selection 

It is common practice to train several networks and then select the best one 
according to the performance on an independent validation set. This procedure 
has the disadvantages as already noted above in the context of determining the 
weighting parameter A. 

Bayesian learning for neural networks does not only allow to regularize the 
model automatically but also provides a quality measure, the so called model 
evidence. On this basis it is possible to select the model with the highest evidence 
from a set of trained networks. This procedure is described below in greater 
detail. 

Forming a committee of networks is another approach to overcome these 
drawbacks and to avoid favouring one model while discarding all others. Several 
methods of forming committees were suggested in recent years. An overview is 
given in the book Combining Artificial Neural Nets ISharkev, U)DD| . 

In this paper we focus on the optimization of a single model and using the 
model evidence as selection criteria. A committee approach that combines proper 
regularization with Bayesian learning and a evolutionary search for (stochasti- 
cally) independent networks is described in |Hagg, 2(11701 . Note that independence 
of committee members is crucial to further improvement of the generalization 
performance. 

2 Neural Networks and Bayesian Learning 

The goal of network training is to estimate a functional relationship based on 
the given data. To minimizes the expected risk, i.e., the generalization error, 
the data-generating process should be approximated. This process is usually 
described in terms of the joint probability p(x, t) = p(t|x)p(x) in input-target 
space. To make a prediction for former, unseen values of x the conditional prob- 
ability p(t|x) is of interest. The neural network is modelling this density. 
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Different assumptions about this density lead to different error functions. 
For example, assuming that the target data is generated from a function with 
additive Gaussian noise, leads to the well known mean square error. In the fol- 
lowing I will focus on regression problems using the mean square error function. 
The theory can although be applied to classification problems as well. For an 
excellent overview, see [Bishop, 1995| . 

The application of Bayesian learning to neural networks was introduced 
by MacKay as a statistical approach to avoid overfitting, [MacKay, 1992a| 
[Bishop, 19951 . In contrast to the sampling theory, where the free parameters 
of probability must be attached by frequencies of samples drawn from a distri- 
bution, the Bayesian statistics uses a priori probabilities to describe degrees of 
beliefs in parameters. The main idea is to combine the sample information and 
the prior belief to form the posterior probability of the parameters. 

Given the training data D we want to find the ’best’ model M{0) with 
parameter vector 0. This is expressed in Bayes’ theorem as follows 

mD) = ( 2 ) 

The best model maximizes the posterior probability, i.e., we want to find 0* 
such that 

p{0*\D) > p{0\D) V6). 

In case of neural networks the parameter vector might consist of the weight vector 
w, the weighting factor A as well as the topology of the network. In general, we 
cannot determine all these parameters at once. For that, the Bayesian framework 
proceeds in a hierarchical fashion. While keeping all other parameters fixed we 
search on the first level for an optimal weight vector, on the second level for 
optimal weighting coefficients and on the third level for the appropriate network 
topology. Gomparing network topology is often done manually by the developer. 
In this paper we suggest to intertwine the Bayesian learning approach with 
an evolutionary search procedure to find an optimal topology and select input 
features in an efficient way. In the following the three levels are described in 
more detail. 

2.1 The Likelihood 

If a training set D = {(xi, ti), . . . , (x^v, t^)} of N input-target pairs is given and 
if we assume that the target data is generated from a function with additive 
Gaussian noise, the probability of observing tm is given by 

p{tm\Xm,^,P) = ^exp{-^{y{Xm,^) ~ tmV), (3) 

where ?/(a;m,w) denotes the output of the neural network with weight vector w 
and j3 controls the noise in the data, i.e., 1//3 is just the variance of the noise. 
Provided the data points are drawn independently, the probability of the data, 
called the likelihood is 
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N 

p{D\w,P)= Y[p{tm\Xm,^,P) 

m—1 



exp ( - | 



N 



Zoifi) 



( 4 ) 



where Z^iP) = is the normalizing constant. It follows that the sum- 

of-squares error of the data E£>{w) = 4 J2{y{xm,'^) — just expresses the 
likelihood function. 



2.2 The Prior Probability 

In the simplest case of Bayesian learning the prior probability distribution of the 
weights is assumed to be Gaussian with zero mean. This restricts the complexity 
of the neural network by searching for small weight values. By denoting 4 as 
the variance of the Gaussian the prior is of the form 



p(yv\a) 



1 

Zw{a) 

1 

Zw{a) 



exp{—aEw) 



w 



exp(--^u;|) 






( 5 ) 



where Ziy(a) = is the normalization factor. 



2.3 The Posterior Distribution of the Weights 

Due to Bayes’ rule © we get from © and © the posterior weight distribution 

p(w|D, a, (5) = ^ eyip{-(3ED - aEw} (6) 

Zs{a,(i) 

Instead of maximizing the posterior it is equivalent but simpler to minimize 
the exponent. This is exactly learning in the sense of equation © with X — 
Usually fast learning algorithms like Scaled Gonjugate Gradient IMpller, 1993| or 
Rprop [Riedmiller, 199^ are used to find a local minimum of the error function. 



2.4 The Evidence for a and j3 

On the first level of the Bayesian framework a method was developed to optimize 
the weights where the so-called hyperparameters a and /3 were assumed to be 
known. Due to the Bayesian approach we are able to determine the hyperparam- 
eters automatically during the training process. This can be done by maximizing 
the probability of a and (3 given the data. 



p(a,(}\D) 



p(D|a,/3)p(g,/3) 

p{D) 



( 7 ) 



The prior for a and (3 are usually assumed to be constant in order to give equal 
importance to all possible values [Bishop, 1995|. Such priors are called to be 
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non-informative. The big disadvantage of uniform priors is that they are not 
invariant under reparametrization. A widely used method to determine non- 
informative priors that are invariant under reparametrization is presented by 
Jeffreys, [Jeffreys, 1961| . Gutjahr calculated Jeffreys prior for the hyperparame- 
ters a and j3 to p{a) = ^ resp. p(/3) = ^ (cf. [Gntjahr, 1998| , and jGutjahr, 1999| 
for an extensive discussion). 

In order to optimize the hyperparameters during the training process, we 
have to maximize the probability of a and /3 given the data according to (0. 
Note that it is assumed, that the weight vector is optimized, i.e., the network 
was trained to a local minimum of the error function. This is denoted in the 
following with , etcQ With Jeffreys priors for a and /3 we get 

logp{a, P\D) cx logp{D\a, /3) -k logp(a, /?) 

= -aE^P - pE^P - i logdet(A) 

W N N 

+ —loga+ y log/3-y log(27r) 

- log a - log P (8) 



where A is the Hessian of the error function and we assumed the hyperparame- 
ters to be independent. The formula is basically the same as in [Bishop, 1995| ex- 
cept for the last two summands (cf. [Bishop, 19951 or |Gutiahr,~^99riKagg, 2U0Ul 
for details of the computation). We calculate the partial derivatives of (0 with 
respect to a and P and determine the optimal values of the hyperparameters by 
setting the results to zero. This provides the following update rules for a and P 



a 



new 



i ^ 1 

2E^P ^ A, + a E^P 



( 9 ) 



and 

1 A 1 

= ( 10 ) 

In comparison to the formulas given in [Bishop, 1995| we see that these update 
rules produce smaller values for the hyperparameters. This does not automati- 
cally mean that A will also get smaller, because the latter depends on the relative 
values of a and p. 

In a practical implementation we have to find optimal values for a and P 
as well as for w at the same time. This is solved by a iterative algorithm that 
periodically re-estimates a and P according to equation (jOJ and equation (HDD, 
after a minimum of the current error function was reached. Note that when re- 
estimating the hyperparamters the error function changes. Figure ^ illustrates 
the iterative algorithm, which has similarities with the basic principle of the 
EM-algorithm. 

^ MP means most probable 
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Oo , Po Ol , Pi , P2 




Fig. 1. The figure shows the error curve during training and the effects of the update 
of the hyperparameters. Initially the are initialized with oo and /3o, than the weight 
vector is optimized by gradient descent. If a minimum of the current error function is 
reached the evidence p(D|ao,/?o) is computed and the update is performed. Note that 
it takes at least two updates until we have with p(D\a\,j3\) a value for the evidence 
that is based on hyperparameters that were computed by the Bayesian approach. 

A great advantage of the Bayesian approach is that a large number of regular- 
ization coefficients can be used since they are determined automatically. Weights 
can be grouped together where each group gets an own hyperparameter a^- That 
is, the error function is changed to 

1 ^ ^ ( \ \ 

E = [i- + ^{ak- - XI 

n—1 k—1 V w£Wk / 

The extension of the Bayesian framework to several weight groups is straight- 
forward [MacKay, 1992b| , [Thodberg, 1993| [JNautze & (dutjahr, . It is com- 
mon to put all the weights between two layers in one group [Sishop, 1995[ 
[Gutjahr, 1999| . In case of small data sets or in case of networks with few hid- 
den units it is sometimes of advantage to use only one weight group as in the 
standard weight decay approach, since only one hyperparameter controlling a 
distribution has to be estimated on the basis of few weights. 

2.5 The Model Evidence 

On the third level we can compare different models, for example, networks with 
different topologies. Using Bayes’ rule we can write the posterior probability of 
a model % as 



P{'H\D) = 



p{D\n)-P{U) 

p{D) 



If we assign the same prior to every model, than it is sufficient to evaluate 
the quantity p{D\TL), which is called the evidence for TL IMacKay, 1992al 
[Bishop, 19951 . Marginalizing over a and (3 provides 

p{D\H) = J J p{D\a, P,'H)p{a, PlV.) da df3 
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The first factor on the right hand side is just the evidence for a and /3 from the 
previous level. Integration over a and (3 is usually done by approximating their 
posterior distributions on the second level by Gaussians. See IBishop, 19951 and 
especially IGutjahr, 19991 for details. The logarithm ol p{D\H) is then given as 

lnp(£)|'H) = -PoptEo - ctoptEw “ ^ det(A) 

N N W 

- y ln(27r) + — IniPopt) + y H<^opt) 

+ 21n^ + ^In (^) + ^In (^) ■ (12) 

Note that the evidence depends on the determinant of the Hessian, det(A); i.e., 
the product of its eigenvalues. This makes the result more sensitive to little 
deviations caused by approximations. Nonetheless, the evidence is usually nega- 
tively correlated with the generalization error ||MacKay, 1992b| [Thodberg, 1993| 
[Ragg fc Gutjahr, 1998| giving us a hint which networks to prefer. 

It is possible to exploit this correlation even further by intertwining a search 
procedure with the iterative Bayesian learning algorithm. By using a popula- 
tion of networks every iteration step can be considered as a generation of an 
evolutionary algorithm. Instead of re-training all networks, the ones with higher 
evidence are allowed to provide more offsprings on expense of the others. That 
means in figure Q] that after two re-estimation steps the values of the evidence 
for the different networks are compared and the ones with lower values are dis- 
carded. In this way the training process is restricted to favourable areas of the 
search space, such that the resulting networks have higher evidence compared 
to the standard training procedure |Kagg, 20(J(J| . In the following I will describe 
the basic evolutionary algorithm and its operators. 

2.6 Evolutionary Search 

The concept of solving optimization problems by means of evolutionary strategies 
was developed in the 1960s by Rechenberg and Schwefel |Itectenberg, 19941 
Pjcbwef'el, 199SI , and was since then used for a variety of problems including the 
optimization of neural networks [Aiander, 1996| . The approach pursued at our 
institute was to use evolution for topology optimization and combine it with 
learning by gradient descent IIBraun fc Ragg, 19961, [Braun, 19971 . Recently, we 
showed that the evolutionary search for sensible topologies and the iterative 
Bayesian learning framework can be intertwined by using the model evidence, 
i.e., its posterior probability, as fitness value [Ragg, 2001)1 . Figures 01 and 0| show 
the dependency of the evidence from the number of hidden units and the number 
of input features. An evolutionary algorithm will tend to find local maxima in 
the shown fitness landscapes. 

Using an evolutionary algorithm as framework for searching an optimal so- 
lution with respect to the fitness function as described below has several advan- 
tages: As every heuristic search algorithm it has to handle the dilemma between 
exploitation and exploration. This is done by parallelizing the search using a 



56 



T. Ragg 



population of networks and stochastic selection of the parents each generation. 
This explorative search is biased towards exploitation by preferring fitter par- 
ents. Furthermore, the algorithm is scalable with regard to computing time and 
performance, i.e., a larger population allows for a more explorative search. 

Incremental elimination of input units does in general not lead to an optimal 
subset of features IFukunaga, 1990[ [Bishop, 19951 . Adding input units during 
the optimization process which maximize the information content can overcome 
these local minima with little computational effort |Ragg fe Gutjahr, 1997} . The 
underlying framework may be summarized as follows: 

Pre-evolution: Initially, a population of p, > 1 networks is created by copying 
a maximal topology that is chosen to restrict the search space. By randomly 
deleting units from the networks and initializing the weights according to the 
assumptions of the Bayesian framework (cf. section 12 . 21 . we start the search 
from different points. These networks are trained by Bayesian learning gradient 
descent as described in section |2 Each parent is assigned its model evidence as 
fitness value. 

Selection: Offsprings are generated after every second re-estimation of the hy- 
perparameters. Each generation A > /i offsprings are generated by first sorting 
the parents into a list based on their fitness, and then selecting A individuals 
with replacement according to a selection function of type x'^ . The parameter 
y is a prefer-factor. If y Ri 1, then all elements have a similar probability to be 
selected. If y gets larger, the probability to select elements from the beginning 
of the list grows accordingly. This mechanism allows us to select fitter networks 
with higher probability. The offspring are identical copies of the parents including 
the values for the hyperparameters (Lamarckism). Note that we use a so called 
(y, A)-strategy tjchwetel, 1995| . This is sensible because the search is intertwined 
with the Bayesian framework. An elitist strategy, i.e., a (y -I- A)-strategy, would 
not serve our purpose since parent networks are not retrained. They would be at 
a disadvantage against offsprings because they run through fewer re-estimation 
cycles. 

Mutation: To exploit the area around a parent model noise is added to the 
weights and hyperparameters of the offsprings. The random values are drawn 
from a normal distribution with zero mean and variance 10“^) and scaled by 
the value of the component, i.e., Wi ^ Wi + Wi * randi). This is useful, because 
several approximations are made in the training process. 

To change the network structure in a sensible way, we need to have a criterion 
to sort the units by saliency and delete low-saliency units. On the other hand, in 
case of addition of units, we should increase the network size only by important 
items. Since the degree of freedom is controlled by the weight-decay regularizer, 
simple and fast computable heuristics are used to optimize the number of hid- 
den units. Weights are not removed to avoid small weight groups, which would 
interfere with the estimation of the hyperparameters in the Bayesian framework. 
Since a regularizer is used, it is not necessary to use advanced (computational 
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intensive) methods that use the Hessian since they do not perform better than 
simple pruning techniques in this case |Kagg et aL, 1997| . Thus, hidden units are 
sorted with respect to the size of the sum of their weights. Adding hidden units 
is done randomly. The most computational effort is used to optimize the input 
structure The input units are sorted with respect to the information content 
I{X;Y) (mutual information) of the resulting input vector about the target: If 

X = {Xi, ■ ■ ■ , Xu-ii Xk+i, ■ ■ ■ , Xi-i,Xi, Ai+i, • • • , Xd) 

is the actual input vector of our neural network than we define 

X(i) ■= (All, . . . , Ai_i, Xi+i, . . . , Xd) 

and 

:= (Xi,...,Xk-i,Xk,Xk+i,...,Xd). 

This means that removes the i-th component of X whereas adds the 
k-th component to X. Every time an input component is to be deleted from resp. 
added to our actual input vector X we c alculate I{X(j);Y) as w ell as I{X^'^'>;Y) 
where i, k take all possible numbers. See IRagg fc Gut.iahr, 19971, IKagg, 2L)L)L)| for 
details of the computation. A nice introduction into information theory can be 
found in [Cover fc Thomas, 199l|, An ov erview on probability density estimation 
procedures is given by T^ . They are used to estimate the mutual 

information between the input vector and the target. Based on this calculation, 
we sort the resulting subsets of input variables by the following criteria: 
Remove: ^ Xj Y) > I{X(^jy, Y) 

Insert: X'= ^ X' I{X^’^^;Y) > /(X®; X) 

For each selected parent one input unit and/or one hidden unit is mutated with 
a certain probability. The candidates for input unit mutation are chosen from 
the sorted list according to a selection function (as above) where y takes a large 
value, e.g., y > 2, i.e, the first 25% of the list are selected at least as often as the 
last 75%. 

Optimization and Evaluation: The resulting offspring networks are then 
trained as described in section 12 The hyperparameters are re-estimated two 
times during the optimization phase. The reason for that is, that the Bayesian 
framework allows us to compute the evidence of the network after the error 
was minimized. The evidence refers to the hyperparameters which were used 
during this cycle. That means, it is not until the second iteration that we can be 
sure that the computation of the evidence is based on a set of hyperparameters 
that were estimated in an optimal way (cf. figP). At last the model evidence is 
assigned to the network as fitness value. 

Survival: The loop is repeated until a user-defined stop criterion is fulfilled, 
e.g., a fixed number of generations, or until the population collapsed into a 
single search point. The algorithm just presented favours in every generation 
networks with higher evidence. That is, the search is with each generation less 
and less explorative and will finally find a (local) optimum of the evidence in 
parameter space. 
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3 Experimental Results 

Two problems are used here to evaluate the approach - a noisy regression problem 
and a classification task from the UCI repository. Results on more problems are 
described in |Ragg, 20001 . 

Noisy sine regression: The data for this benchmark was generated by adding 
Gaussian noise to a sine function (Figure Et). 40 data points were used for 
training the model while 100 additional data points (from the underlying process) 
were used to compute the test error. FigureEb shows several models for this data 
set, two of them trained with Bayesian learning, which show sensible yet still 
different approximations of the underlying process. These dependency of the 
results on the initialization and the network topology is visualized in figure 13 
Several thousand networks were trained to establish this relationship. We clearly 
observe that there is an optimal network topology with about 15 to 20 hidden 
units. The correlation between the evidence and the test error was p = —0.35. 
An evolutionary optimization using the evidence as fitness value will now tend 
to find a maximum of the model evidence in parameter space which can clearly 
be seen in figure 03. The average error of all trained networks is 0.04 ± 0.08 
which reduces to 0.014 ± 0.012 if we train 100 networks with randomly chosen 
topology and select the model with highest evidence. Evolutionary optimization 
further reduces the test error to 0.011 ± 0.003. All improvements are significant 
according to the t-test (threshold to. 95 ;io = 1-82); see jBuning fc Trenkler, 1994| 
for statistical test procedures. 

Thyroid classification: In the problem described above the input consisted 
of only one neuron, i.e. feature selection was not necessary. The thyroid clas- 
sification problem is form the UCI repository and contains 21 input features. 
About 7200 patterns are available, from which we used only 600 for training the 
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Fig. 2. The figure shows the data for the benchmark regression problem, a) The under- 
lying process is just the sine function sin{x). The target data was generated by adding 
Gaussian noise with variance 0.4 and zero mean. Note that there is an interval, where 
no data is available, b) Output functions of 4 models are plotted: An overregularized 
network, an underregularized network and two proper regularized networks trained 
with Bayesian learning. 
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(a) (b) 





(c) (d) 



Fig. 3. The figure shows the dependency of the evidence from the number of hidden 
units and the number of input features, which was determined by the training of several 
thousands of networks, a) Average, minimal and maximal evidence. The networks with 
the highest evidence have between 16 and 24 neurons in the hidden layer, b) Range of 
average and maximal evidence. The crosses (+) correspond to the networks with the 
highest evidence (of ten trials of training 50 networks). The other symbols (x) mark 
the results for the evolutionary optimization. They are all concentrated in the area of 
the maximum, c) Average, minimal and maximal test error, d) Range of average and 
minimal test error. Symbols used as in (b) 



networks and the other 6600 solely to estimate the generalization performance. 
The task is to classify a malfunction of the thyroid (under, over or normal). 
According to table 0 we should not exceed an input dimensionality of 5 or 6 
features, if we want to learn a non-linear relationship. Figure shows clearly 
a region with maximal evidence (left bottom corner) which is highly correlated 
with the generalization error (p = —0.90). The key problem here is to find an 
optimal subset of input features. Note that this is a combinatorical problem 
(different from optimizing the number of hidden features) . For the given fitness 
landscapes the proposed evolutionary algorithm performs a gradient ascent to 
the region with maximal evidence. This is possible since Bayesian learning reg- 
ularizes the networks properly and the mutation of input units based on mutual 
information leads so sensible subsets of features with a high information content. 
The error rate of all trained networks is 2.8±0.01% which reduces to 2.6 ±0.1% 
if we train 100 networks with randomly chosen topology and select the model 
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Fig. 4. The figure shows the dependency of the evidence resp. the generalization error 
from the number of hidden units and the input features. About 10000 networks (with 
random topology) were trained to establish this dependency. Only 10 input features 
are shown for a better visualization, a) Model evidence over parameter space. A clear 
maximum is observable for 4 to 6 input features and 2 to 6 hidden neurons, which 
can easily be found by a evolutionary search procedure if we start with random search 
points in this parameter space. The rhombuses mark the results of 10 evolutionary 
optimization runs, from which some lead to identical network structure. All models 
group in the region of high evidence, b) Generalization error over parameter space. 
The evidence is highly correlated with the generalization error (p = —0.90). 

with highest evidence. Evolutionary optimization further reduces the test error 
to 1.8 ± 0.3%. All improvements are significant according to the t-test. 



4 Conclusions 

In this paper an approach was presented that integrates several important steps 
of neural network design into an evolutionary optimization procedure. This 
method was primarily aimed at developing models for tasks where the data is 
noisy and limited. Bearing in mind the curse of dimensionality we should select 
few features with a high information content, thus approximating the function 
being searched in a low-dimensional space. Furthermore, it is difficult to decide 
which part of the data should be used for training. This problem can be tack- 
led by integrating over several samples of the data set. Moreover, in a practical 
application we also face the problem of model selection. That is, having trained 
several networks, we would like to select one that performs significantly better 
than average. 

We started out be reviewing the some problems that arise when trying to 
estimate a functional relationship with neural networks. Learning from empirical 
data is an ill posed problem making it necessary to use a proper regularization 
method, e.g., weight decay. A crucial problem is to determine the weighting fac- 
tor of the regularizer. This can best be done by applying a Bayesian learning 
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algorithm, because it adapts this parameter automatically without the necessity 
of additional validation data. Furthermore, it provides a quality measure, i.e., 
the model evidence, which can be used for model comparison. Furthermore, the 
evidence can serve as the optimization criterion of an evolutionary algorithm 
(cf. IRagg fc Gutjahr, 1998| ). By this, the trial and error search for models with 
high posterior probability can be replaced by a more efficient search procedure. 
Moreover, feature selection and topology optimization can be intertwined with 
the network training such that a local maximum of the fitness function over 
parameter space can be found. For two problems it was shown that the opti- 
mization procedures finds regions with higher model evidence thus lowering the 
risk of choosing a wrong model and improving significantly the generalization 
performance of the models. 

Acknowledgments. This work was supported by the Deutsche Forschungsge- 
meinschaft (DFG), ME 672/10-1, Tntegrierte Entwicklung von Komitees neu- 
ronaler Netze’. 

References 

[Alander, 1996] Alander, J. T. An Indexed Bibliography of Genetic Algorithms and 
Neural Networks. Technical Report 94-1-NN, Department of Information Tech- 
nology and Production Economics, University of Vaasa, 1996. 

[Bishop, 1995] Bishop, C. M. Neural Networks for Pattern Reeognition. Oxford Press, 
1995. 

[Braun & Ragg, 1996] Braun, H. and Ragg, T. ENZO - Evolution of Neural Networks, 
User Manual and Implementation Guide, http://illwww.ira.uka.de. Technical 
Report 21/96, Universitat Karlsruhe, 1996. 

[Braun, 1997] Braun, H. Neuronale Netze: Optimierung durch Lernen und Evolution. 
Springer, Heidelberg, 1997. 

[Buning & Trenkler, 1994] Biining, H. and Trenkler, G. Nichtparametrische statistis- 
che Methoden. de Gruyter, 1994. 

[Cover & Thomas, 1991] Cover, T. and Thomas, J. Elements of Information Theory. 

Wiley Series in Telecommunications. John Wiley & Sons, 1991. 

[Fukunaga, 1990] Fukunaga, K. Introduction to Statistical Pattern Recognition. Aca- 
demic Press, 1990. 

[Gutjahr, 1998] Gutjahr, S. Improving the determination of the hyperparameters in 
bayesian learning. In Downs, T., Frean, M., and Gallagher, M., editors. Proceed- 
ings of the Ninth Australian Conference on Neural Networks (ACNN 98), pages 
114-118, Brisbane, Australien, 1998. 

[Gutjahr, 1999] Gutjahr, S. Optimierung Neuronaler Netze mit der Bayes’schen Meth- 
ode. Dissertation, Universitat Karlsruhe, Institut fiir Logik, Komplexitat und 
Deduktionssysteme, 1999. 

[Hertz et al, 1991] Hertz, J., Krough, A., and Palmer, R. G. Introduction to the theory 
of neural computation, volume 1 of Santa Pe Institute, Studies in the sciences of 
complexity, lecture notes. Addison- Wesley, 1991. 

[Jeffreys, 1961] Jeffreys, H. Theory of Probability. Oxford University Press, 1961. 
[Krogh & Hertz, 1992] Krogh, A. and Hertz, J. A Simple Weight Decay Can Improve 
Generalisation. In Advances in Neural Information Processing 4, pages 950-958, 
1992. 



62 



T. Ragg 



[MacKay, 1992a] MacKay, D. J. C. Bayesian interpolation. Neural Computation, 
4(3):415-447, 1992. 

[MacKay, 1992b] MacKay, D. J. C. A practical bayesian framework for backpropaga- 
tion networks. Neural Computation, 4(3):448-472, 1992. 

[Mpller, 1993] Mpller, M. A Scaled Conjugate Gradient Algorithm for fast Supervised 
Learning. Neural Networks, 6:525-533, 1993. 

[Nautze & Gutjahr, 1997] Nautze, C. and Gutjahr, S. Extended bayesian learning. In 
Proceedings of ESANN 97, European Symposium on Artificial neural networks, 
Bruges, pages 321-326, 1997. 

[Ragg & Gutjahr, 1997] Ragg, T. and Gutjahr, S. Automatic Determination of Opti- 
mal Network Topologies based on Information Theory and Evolution. In IEEE, 
Proceedings of the 23rd EUROMICRO Conferenee 1997, pages 549-555, 1997. 

[Ragg & Gutjahr, 1998] Ragg, T. and Gutjahr, S. Optimizing the Evidence - with 
an application to Time Series Prediction. In Proceedings of the ICANN 1998, 
Sweden, Perspectives in Neural Gomputing, pages 275-280. Springer, 1998. 

[Ragg et al, 1997] Ragg, T., Braun, H., and Landsberg, H. A Comparative Study of 
Neural Network Optimization Techniques. In Lecture Notes in Computer Scienee, 
Proceedings of the ICANNGA 1997, Norwieh, UK, pages 343-347. Springer, 1997. 

[Ragg, 2000] Ragg, T. Prohlemlosung durch Komitees neuronaler Netze. Dissertation, 
Universitat Karlsruhe, Institut fiir Logik, Komplexitat und Deduktionssysteme, 
2000 . 

[Ramsay & Silverman, 1997] Ramsay, J. O. and Silverman, B. Functional data analy- 
sis. Springer, 1997. 

[Rechenberg, 1994] Rechenberg, I. Evolutionsstrategie ’94- Frommann-Holzboog Ver- 
lag, Stuttgart, 1994. 

[Riedmiller, 1994] Riedmiller, M. Advanced supervised learning in multi-layer percep- 
trons - from backpropagation to adaptive learning algorithms. Int. Journal of 
Computer Standards and Interfaees, 16:265-278, 1994. Special Issue on Neural 
Networks. 

[Ripley., 1996] Ripley., B. D. Pattern recognition and neural networks. Cambridge 
University Press, 1996. 

[Schwefel, 1995] Schwefel, H.-P. Evolution and Optimum Seeking. Sixth-Generation 
Gomputer Technology Series. John Wiley & Sons, 1995. 

[Scott & Thompson, 1983] Scott, D. and Thompson, J. Probability density estimation 
in higher dimensions. In Gentle, J., editor, Computer Scienee and Statistics: 
Proceedings of the Fifteenth Symposium on the Interface, pages 173-179. 1983. 

[Sharkey, 1999] Sharkey, A. J. Multi-Net Systems. In Sharkey, A. J., editor. Combining 
Artificial Neural Nets, pages 1-30. Springer, 1999. 

[Silverman, 1986] Silverman, B. Density Estimation for Statistics and Data Analysis. 
Ghapman and Hall, 1986. 

[Thodberg, 1993] Thodberg, H. H. Ace of bayes: Applications of neural networks with 
pruning. Technical Report 1132E, Danish Meat Research Institute, 1993. 

[Vapnik, 1982] Vapnik, V. Estimation of Dependences Based on Empirieal Data. 
Springer, 1982. 

[Vapnik, 1995] Vapnik, V. The Nature of Statistical Learning Theory. Springer, 1995. 

[White, 1989] White, H. Learning in artificial neural networks: a statistical perspective. 
Neural Computation, 1:425-464, 1989. 




Abductive Partial Order Planning with Dependent 

Fluents 



Liviu Badea and Doina Tilivea 

AI Lab, National Institute for Research and Development in Informatics 
8-10 Averescu Blvd., Bucharest, Romania 
badea@ici . ro 



Abstract. Our query planning application for system integration requires a 
backward partial-order planner able to deal with non-ground plans in the pres- 
ence of state constraints. While many partial-order planners exist for the case of 
independent fluents, partial-order planning with dependent fluents is a signifi- 
cantly more complex problem, which we tackle in an abductive event calculus 
framework. We show that existing abductive procedures have non-minimality 
problems that are significant especially in our planning domain and propose an 
improved abductive procedure to alleviate these problems. We also describe a 
general transformation from an abductive framework to Constraint Handling 
Rules (CHRs), which can be used to obtain an efficient implementation. 



1 Introduction and Motivation 

The integration of hybrid modules, components and software systems, possibly 
developed by different software providers, is a notoriously difficult task, involv- 
ing various extremely complex technical issues (such as distribution, different 
programming languages, environments and even operating systems) as well as 
conceptual problems (such as different data models, semantic mismatches, etc.). 

Solving the conceptual problems requires the development of a common, ex- 
plicit, declarative knowledge model of the systems to be integrated. Such a model 
should be used (by a so-called mediator) not only during the development of the 
integrated system, but also during runtime, when it can be manipulated by an 
intelligent query planning agent to solve problems that could not have been 
solved by any of the specific information sources alone and might even not have 
been foreseeable by the system integrator. Since in most realistic applications, 
the state of the databases or of the procedural components changes as a problem 
is being solved, we shall describe the services offered by such procedural appli- 
cations and the database updates as actions. 

The query planner of the mediator transforms a user query into a partially ordered 
sequence of information-source specific queries, updates and calls to application inter- 
faces that solve the query. The main requirements which our system integration appli- 
cation imposes on the query planner are the following: 
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• It should be a partial-order planner in order to take advantage of the intrinsic 
distributed nature of the integrated system. 

• As the information content of the sources (for example databases) can be 
quite large, forward propagation of the initial state of the sources is impossi- 
ble in practice. We therefore need to develop a backward planner, which 
would ensure minimal source accesses with maximally specific queries. 

• The planner should reason with arbitrary logical constraints between state 
predicates (dependent fluents). 

• It should also be able to manipulate plans with variables. UCPOP-like plan- 
ners can do this, but UCPOP only deals with independent fluents. 

Also, although very fast general purpose planners like Graphplan and SATPLAN are 
currently available, these are not usable in our system integration application, where 
generating a grounding of the planning problem is inconceivable (a database may 
contain an enormous number of different constants), while both SATPLAN and 
Graphplan generate a grounding of the planning problem. The same holds for recent 
planners based on Answer Set Programming developed in the Logic Programming 
community, which use efficient propositional answer set generators, like dlv or smod- 
els. 

Unfortunately, there is no implemented planner available with the characteristics re- 
quired by our system integration application. In this paper we describe the construc- 
tion of such a planner. 

The following simple example illustrates the type of problems we are dealing with. 
Assume that the dean of a university plans to assign a high responsibility course (this 
course being assignable only to faculty members). This can be done by applying ac- 
tion assign_course having effect course_assigned and no explicit preconditions. As- 
sume there exist constraints, such as that this course cannot be assigned to a person 
who is not a professor 

^ holds(course_assigned, T), holds(— iprofessor, T) (1) 

or who is not employed by the university 

^ holds(course_assigned, T), holds (—lemployed, T) (2) 

Three additional actions can be used to achieve the final goal: 

• hire (with precondition — lemployed and effects faculty and —iprofessor) hires a 
person that is not registered in the personnel database as employed (this per- 
son may not be directly hired as a professor), 

• promote (with precondition —iprofessor and effect professor) which promotes a 
non-professor to the position of professor, 

• register (with precondition — lemployed and effect employed) which registers a 
person as being employed in the database of the personnel department. 
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The final goal holds(course_assigned,T), holds (faculty,!) can be achieved by the par- 
tial-order plan (constructed by our algorithm) presented in the Figure above. Double 
arrows denote protection intervals not_cKpped(Ti, F, Tf), while simple arrows are plain 
ordering constraints. Note that state constraints induce implicit preconditions as well 
as ordering constraints and protection intervals. For example, the ordering constraints 
r < a_c and pr < a_c have been posted to prevent the activation of the state constraints 
(2) and (1) respectively. 



2 Problems with Abductive Procedures 



Partial-order planning can be viewed as abductive planning in the Event Calculus 
[7]. Considering the fact that there are many partial-order planners for independ- 
ent fluents, and since adding state constraints to an Event Calculus action specifi- 
cation seems straight-forward (see (8) below), it might seem that it should be easy 
to develop a partial-order planner dealing with dependent fluents]] This impres- 
sion is however misleading: integrity constraints represent a significant compli- 
cation. Intuitively, fluent dependencies seriously complicate the detection of 
‘threats’. As far as we know, there are no implemented sound and complete par- 
tial-order planners able to deal with non-ground plans (i.e. plans with existen- 
tially quantified variables) and dependent fluents exist.0 (Dependent fluents are 
fluents subject to integrity constraints. As usual, integrity constraints may have 
universally quantified variables.) 

In the following we illustrate problems faced by existing abductive procedures, es- 
pecially in our planning domain. 

We start with the following (simplifiec|]l Abductive Logic Programming (ALP) 
specification of the Event Calculus with dependent fluents: 



P" 

r 



holds(F,T) starts(F,T(i), Tn<T, not cUpped(Tn,F,T) 
clipped(To,F,T) <- starts(-iF,Ti), Tn<Ti, Ti<T 
starts (F,0) <r- initially (F) 
starts(F,T) <r- initiates(A,F), happens(A,T) 

^ happens(A,T), precondition(A,F), not holds(F,T) 
^ holds(Fi,T), . . ., holds(Fn,T) 



(3) 

(4) 

(5) 

(6) 
(V) 
( 8 ) 



Abducibles A= {happens} 

Query: ?- holds(F„T), . . ., holds(F„Tj), . . . T,<T„ . . . 



where P are program clauses, while I are integrity constraints. A denotes actions, 
T time points and F fluent literals, i.e. (negated) atoms. starts(F,T) means that 



^ Some implementations (including UCPOP) distinguish between primitive and derived fluents. 
(Primitive fluents are assumed independent and actions cannot have derived fluents as effects.) 
This is a simple form of ramification - the primitive fluents are still independent (state con- 
straints are not allowed). 

^ For example, the planner of [7] is unsound in the presence of state constraints (see the last 
example in section 6 of [7]). [1] has non-minimality problems (see below) and we haven’t been 
able to use it for dependent fluents. 

3 More complicated action languages can be easily incorporated. 
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fluent F becomes true at T, while starts (—iF,T) means that F is terminated at T. 
Direct effects are given by initiates, while preconditions are given hy precondition. 
( 8 ) represents a problem-dependent integrity constraint involving fluents Fi, 

Fn. 

A purely backward abductive procedure (like ACLP [ 4 ] or SLDNFA [ 2 ]) typically 
faces problems in recognizing that an abducihle selected for solving one suhgoal also 
solves a second subgoal. If the second subgoal can be achieved in several ways, the 
first solution returned by the backward abductive procedure might be non-minimal, 
since it might reachieve an already achieved goal. For example, consider the fol- 
lowing ALP P = {p a, q <— b, q <— a} with a, b abducibles and the query ?- p, q. 
For solving the first suhgoal, p, the abductive procedure will abduce a. Then, when 
trying to solve q, it will abduce b, not realizing that q has already been solved by as- 
suming a. (Both ACLP and SLDNFA have this problem, the latter even if a, b are de- 
clared strong abducibles.) 

This problem could be solved by forward propagation of a to q: a => q. Now, when 
trying to solve q, the procedure would find that q has already been achieved. 

Such situations occur very frequently in our planning domain, where an effect can 
be achieved by several actions. For example, if {initiates(a2,f2), initiates(ai,fi), initi- 
ates(ai,f2)}, then the first answer to the query ?- holds(fi,t), holds(f2,t) will be non- 
minimal (both ai and a2 will be applied). 

There is a second situation in which existing abductive procedures might produce 
non-minimal solutions, namely the presence of negative literals in negative goals. As 
these are in fact disjunctions in disguise, existing abductive procedures will treat them 
by splitting, even in cases in which the negative goal is already achieved. For exam- 
ple, for the ALP P = {p <— a, q b, r <— b} with abducibles A = {a, b} and integrity 
constraints I = { ^ not p, not q }, the query ?- r would first lead to abducing b for 
solving r, and then generate the negative goal not p, not q. Because b entails q, this 
goal is already solved, but a backward abductive procedure would not know this and 
would split the negative goal into the positive goal p and the negative goal <r- not q 
(which will be reduced to qV Lfnfortunately, this would abduce a to achieve p, leading 
to a non-minimal solution.^ (In other words, we are “repairing” an already repaired 
integrity constraint.) Again, propagating b forward to q would allow us to inactivate 
the negative goal <r- not p, not q before splitting it. 

This situation does not arise in planning with independent fluents (i.e. without state 
constraints). Several negated literals can appear in negative goals only due to the state 
constraints ( 8 ) (after unfolding holds to not cKpped). The presence of state constraints 
therefore complicates partial-order planning to a significant extent. (The solution to 
the frame problem offered by partial-order planning is especially simple in the case of 
independent fluents.) 

For example, consider the actions {initiates(ai,— ip), initiates (a2,—iq), initiates(a2,r), initi- 
ates(bi,p), initiate s(b2,q), initiates(b3,s)}, the state constraint 

holds(p,T), holds(q,T), holds(s,T) 



4 



SLDNFA has this problem too. Due to its syntactical restrictions, ACLP cannot even deal 
directly with this example. 
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and the goal ?- holds(r,tfin),---- Further assume that actions U2, bi, b2, bs solve all 
positive goals and have already been placed in the plan: happens (a2,t), hap- 
pens(bi,ti), happens(b2,t2), happens(b3,t3), together with the ordering constraints h<t3, 
t2<t3, t2<t, t<t3. Now, the integrity constraint will he successively unfolded to the 
negative goals: 

<r- happens(bi,Ti), not chpped(Ti,p,T), Ti<T, 
happens (b2,T2), not cUpped{T2,q,T), T2<T, 
happens (b3,T 3), not cUpped(T3,s,T), T3<T. 
not cHpped(ti,p,T), not cHpped(t2,q,T), not chpped(t3,s,T), 

T>max(ti,t2,t3). 

(the last step corresponds to the resolution with the ahducibles happens(bi,ti)). 

The last negative goal is already solved because the presence of a2 in the plan 
ensures that cHpped(t2,q,t3) holds. Solving the negative goal by splitting, for exam- 
ple by generating the positive subgoal ?- cHpped(ti,p,T), T>max(ti,t2,t3) would place 
the additional unnecessary action ai in the plan. 

A careful analysis of existing abductive procedures shows that both problems men- 
tioned above involve reachieving an already achieved goal and are related to the 
treatment of implicit disjunctions by splitting. 

We argue that in both cases we should avoid splitting disjunctions when these are 
already achieved. This will not guarantee the minimality of the first solution, but it 
will at least avoid reachieving already achieved goals. Of course, the minimal solution 
is in the search space, but in general we cannot guarantee obtaining it in polynomial 
time. More precisely, the problem of finding a (locally) minima^] explanation in an 
abductive problem with integrity constraints is AP-complete, even in the propositional 
case (Theorem 4.5 of [10]). 

For example, considering the same ALP program P = {p a, q ^ b, r b} as 
above, the query ?- q, p will lead in our framework to the non-minimal abductive ex- 
planation b,a. Note however, that the minimal solution a is in the search space and will 
be found upon backtracking. 



3 Propagating Abductive Changes 

We have seen that the planning problem can be formulated as an abductive prob- 
lem. The main efficiency problem faced by all implemented abductive procedures 
is avoiding testing all integrity constraints after each abductive change. Since the 
integrity constraints have been tested before the change, we should retest only the 
ones that are influenced by the change in some abducible. For example, for 
achieving this, ACLP [4] requires each integrity constraint (IC) to contain at least 
an abducible predicate. The current implementation also requires each non- 
abducible positive condition in an IC not to depend on abducible predicates. If it 
does, as it is usually the case, the user would have to unfold the predicate in the 
IC with its definition until its dependence on the ahducibles is made explicit. 
These strong requirements are needed so that there are only direct influences of 
changing ahducibles on ICs. If this limitation is removed, then we need to be able 



^parsimonious in the terminology of [10]. 
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to determine which predicates are (indirectly) influenced by a change in an ab- 
ducible. This can be achieved by forward propagation of the abductive changes 
from abducibles to other predicates occurring in ICs. 

In the following we propose a mixed abductive procedure combining backward goal 
reduction rules with forward propagation rules for the abductive changes. In this 
problem solving strategy, the goals are reduced backwards to abducibles and con- 
straints, which are then propagated forward (“saturated” to a complete solution). The 
role of the forward propagation rules is not only to detect inconsistencies, but also to 
repair any potential inconsistencies (by adding new abducibles and/or constraints). 
Instead of retesting all ICs after each modification, we propagate the change through 
the ICs and suggest repairs that ensure that the ICs are not violated. 



3.1 Constraint Handling Rules 

Constraint Handling Rules (CHRs) [3] represent a flexible approach to develop- 
ing user-defined constraint solvers in a declarative language. As opposed to typi- 
cal constraint solvers, which are black boxes, CHRs represent a 'no-box' approach 
to CLP. CHR propagation rules are ideal candidates for implementing the rules 
for forward propagation of abductive changes. 

CHRs can be either simplification or propagation rules. 

A simplification rule Head o Guard | Body replaces the head constraints by the 
body provided the guard is true (the Head can contain multiple CHR constraint at- 
oms). 

Propagation rules Head => Guard | Body add the body constraints to the constraint 
store without deleting the head constraints (provided the guard is true). A third, hybrid 
type of rules, simpagation rules Headi \ Head 2 O Guard | Body replace Head 2 by 
Body (while preserving Headi) if Guard is true. (Guards are optional in all types of 
rules.) 



4 Transforming ALPs into CHRs 

In the following, we present a general transformation from an Abductive Logic 
Program (ALP) into a set of Constraint Handling Rules (CHRs), which function 
as an abductive procedure for the given ALP. We also illustrate the transforma- 
tion on the example of partial-order planning with dependent fluents. 

We start from an ALP (PAT)- In the case of partial-order planning, we will use the 
ALP (3)-(8) from Section 2. Our goal is to replace the integrity constraints (ICs) 
(which would naively have to be retested after each abductive step) by forward rules 
for propagating abductive changes. This is useful both for detecting inconsistencies 
and for suggesting repairs. However, not every predicate in an IC can be the target of 
forward propagation - such predicates would have to be unfolded (backwards) until 
"forward" predicates are reached. 
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Since the specific problem may require certain predicates or rules to be "backward" 
— even if they could in principle be "forward" — we allow them to be explicitly de- 
clared as "/jockwarr/ ".[](This is obviously a problem-dependent specification.) 

The transformation rules below will automatically determine the status (for- 
ward/backward) of the predicates and rules that are not explicitly declared as "back- 
ward". 

Note that the transformation of ALPs into CHRs presented below is completely 
general (it works for any ALP program, not just for our planning domain). 

m First, we determine which rules and predicates can be treated by forward propa- 
gation (or, in short, are "forward"). (Rules and predicates which are not "forward" are 
called "backward" and will have to be treated by unfolding.) 

• Abducibles (like happens in our planning domain) are automatically "forward". 
The occurrences of such abducibles p in rule bodies will be replaced by con- 
straints Cp. (Intuitively, Cp denotes the "open" part of predicate p. Technically, 
Cp is used to trigger the forward propagation rules for p — see (*) below). 

• An ALP rule can be "forward" only if all its body literals are "forward". In particu- 
lar, rules with negative literals in the body cannot be used as forward rules (for ex. 
rule (3)). 

• Predicates which appear in the head of at least one "forward" ALP rule are them- 
selves "forward". 

[U Then, we replace the ICs by forward propagation rules. For each IC, we unfold 
the positive literals - in all possible ways with their "backward" rules only - until we 
are left with positive "forward" literals, negative literals or constraints: 

<— pi, ..., pn, not qi, ..., not qm, Ci, ..., Ck 
Such an unfolded IC is replaced by the forward propagation rule 

Cpi, . . ., Cpn => -Cl ; . . . ; -Ck ; c,(qi ; . . . ; qm) (*) 

where -c, is the complement of the constraint c, and c is the conjunction ci, ..., Ck 
(we apply a sort of semantic splitting w.r.t. the constraints). 

This forward rule exactly captures the functioning of the abductive procedure, 
which waits for pi, ..., p„ and only then treats the remaining body by splitting. Note 
that the subgoal q, is propagated (rather than the constraint Cq,). Solving the subgoal q, 
amounts to constructively ensuring that the IC is not violated (this functions as a con- 
structive "repair" of a potential IC violation). 

\H Finally, we replace negative literals not p in bodies of "backward" rules by con- 
straints Cnot o (whose role will be to protect against any potential inconsistencies 
with some p)JjFor each such negative literal we add the IC <— not_p, p, which will 
be treated as in step (2) above. 



® For example, in our planning domain, we may wish to avoid propagating the initial state 
forward, especially if we are dealing with a database having a huge number of records. 

^ Unlike "normal" abducibles which are implicitly "minimized", abducibles of the form Cnot_p 
are subject to a maximization policy. Thus, we cannot expect all instances Cnot_p to be ex- 
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In our planning domain, rule (3) is backward because it has a negative literal (not 
cHpped) in the body. 

Rules with constraints!] in body could be used as forward rules, but they would 

propagate disjunctions (treated by splitting), which should be avoided. (4) will there- 
fore be treated backward because of the constraints '<' in its body. 

(5) will be treated backward because propagating the initial state of a database for 
example (having a huge number of records) is infeasible in practice. 

(6) can be treated as forward, so starts will be a "forward" predicate: 

Chappens(A,T), initiates (A,F) =5> Cstarts(F,T) (9) 

Similarly, the IC (7) induces the forward rule 

Chappens(A,T), precondition(A,F) => holds(F,T) (10) 

(The action description predicates precondition and initiates are "static con- 
straints", i.e. constraints that are given at the beginning of the problem solving 
process and which do not change.) 

Finally, the ICs (8) related to state constraints are unfolded (since holds is a back- 
ward predicate) to 

<r- starts(Fi,Ti), not chpped(Ti,Fi,T), Ti<T, ..., 
starts(F„,Tn), not cUpped(Tn,F„,T), T„<T. 
which induce the forward rules 

Cstarts(Fi,Ti), ..., Cstarts(Fn,Tn) => T>max(Ti, ...,T„), (11) 

( chpped(Ti,Fi,T) ; ... ; chpped(T„,F„,T) ). 

(The first disjunct, T<max(Ti, ...,T„), of the consequent could be dropped since it 
mentions the free variable T.) 

Since starts also has a backward rule (5), the IC (8) also unfolds to 
initially (Fik), not cUpped(0,Fik,T), . . ., 

. . ., starts(F,i,T,i), not cHpped(T,i,F,i,T), T,i<T, . . . 

The induced propagation rule is 

. . ., Cstarts(Fji,T,i), ... => . . . ; ~initially(Fi) ; . . . ; (12) 

initially(F;i), ..., initiaUy(F„n), T>max(Tji, ...), 

[ . . . ; chpped(0,Fi,T) cUpped^,i,F,i,T) ; . . . ] 

Finally, the IC 

<r- not_chpped(To,F,T), cKpped(To,Fi,T), F=Fi 
is unfolded with (4) to 

<r- not_chpped(To,F,T), starts(— iFi,Ti), To<Ti, Ti<T, F=Fi 
and, since starts still has a "backward" rule (5), also to 
<r- not_chpped(To,F,T), initiaUy(Fi), Tn<0, 0<T, F=Fi 



plicitly propagated and we should therefore avoid having to forward propagate Cnot_p. not_p 
will thus be a backward predicate, used just to avoid violations of the IC <— not_p, p. 

^ Here, by constraints we mean predicates for which the Closed World Assumption does not 
hold. For example, the absence of T1<T2 from the constraint store does not entail T1>T2. 

^ Having both backward and forward versions of rule (6) does not lead to redundancies or 
loops. The role of the backward rule is to reduce a goal formulated in terms of staits(F,T) to 
assuming the abducible Chappens(A,T) for an action A that initiates F. Then the forward rule 
propagates all other effects of A (not just the one that triggered the action application). 
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The latter IC can be dropped since To<0 is inconsistent with the general con- 



straint 0<To. The forward propagation rule induced by the first IC is 

Cnot_clipped(To,F,T), Cstarts(— iF,Ti) F^^tpi ; F=Fi,(To>Ti ; Ti>T) (13) 

We have thus obtained the following set of rules 

Backward goal reduction rules 

holds(F,T) O starts(F,Tn), To<T, Cnot_cKpped(To,F,T) (14) 

starts(F,T) O initiaUy(F), T=0 ; initiates (A,F), Chappens(A,T) (15) 

cHpped(To,F,T) O starts(— iF,Ti), To<Ti, Ti<T (16) 

Forward propagation rules: (9)-(12). 

We also have a general rule for all constraint predicates p: 

Cp(Xi) \ p(X2) O Xi=X2 ; Xi;^X2, p(X2) (17) 



which is given a higher priority than the other rules and which tries to solve a 
goal p(X 2 ) by reusing an already existing constraint Cp(Xi) (propagated earlier by 
a forward rule). This rule also leaves the alternative of constructively achieving 
p(X 2 ) open. 



4.1 An Improved CHR Implementation 

While the above approach avoids reachieving already solved positive goals, it 
doesn’t avoid splitting when dealing with negative literals in negative goals (in 
our case ’not clipped’ in negative goals originating from state constraints). An 
improved implementation would have to explicitly represent the disjunctive goals 
(involving cUpped) before actually splitting them. 

We shall represent partially activated state constraints as ic(Head Body) (the ini- 
tial state constraints have the form ic(fail <r- Body)). Like in rule (11), such integrity 
constraints are (partially) activated by Cstarts(F,T) constraints: 

Cstarts(Fi,T), ic(Head ^ F 2 , Body) => Fi= 3 p 2 , ic(Head ; — 1 F 2 1 T 4— Body) (18) 

; ~(Fi=,F2) 

where — iF | T denotes the fact that the IC has been activated by a fluent F becom- 
ing true at time point T (Fi=gF 2 is defined in Section 5.2 below). 

The following rule inactivates an IC if a pair of its activating starts literals is 
clipped: 

Cstarts(-,F,T), T,<T, T<T, \ ic(Head;-,F, | T,;-,F, | T, ^ Body) O (1 9) 

F=F, ; ~(F=F,), ic(Head;-,F, | T,;-,F, | T, Body) 
(Note that this rule has the form CcUpped \ ic <=7> true, or even 
Cclipped \ (clipped ; . . . ; clipped) o true.) 

Finally, if an IC has been completely activated (without being inactivated by the 
previous rule, which has higher priority), then we should clip at least a pair of starts 
literals that activated it: 

ic(Head <r- true) => ic_cHp(Head) (20) 

Note that it is sufficient to clip a pair (— iFi|Ti, — iF, |T,) such that T, has no known 
antecedent (w.r.t. the temporal order) in the set of activators, while T, has no 
known successor: 
ic_cHp(Head) : - 

lower_bound(FIead, — iF, | T,), upper_bound(Head, — iF, | T,), cHpped(Ti, F,, T,). 
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We also have to deal with the possibility of ICs being activated by the initial state 
(while avoiding the forward propagation of the initial state): 

ic(Head ^ Body) => yora//Body = Fi, Fk 7/»«7initiaUy(Body) (21) 

ic(— iFi I 0 ; . . . ; — iFk | 0 ; Head true). 

The ICs propagated by these rules can of course be inactivated by the previous 
inactivation rule (19). (Note that in the improved approach rules (18)-(21) replace 
rules (11) and (12).) 

The above rules have been directly implemented in the ECLiPSe as well as SICStus 
CHR environments. We have run tests comparing a simple partial order planner for 
independent fluents (similar to UCPOP and SNLP) with the planner described above 
and noticed no overheads (due to the treatment of dependent fluents) on planning 
problems with independent fluents. Since there are no other planners dealing with non- 
ground plans and dependent fluents, no standard benchmarks are currently available. 
However, we have successfully tested the planner on query planning problems in 
system integration, where depedent fluents occur naturally (as briefly described in the 
Introduction). 



5 A General Abductive Procedure 

In order to better clarify the relationship of our approach to existing abductive 
procedures, we present in the following a general abductive procedure that tack- 
les the above-mentioned non-minimality problems by allowing a limited form of 
forward reasoning in addition to backward goal-directed reasoning. The proce- 
dure doesn’t aim at improving the implementation from Section 4.1, its main role 
being to generalise the approach from the previous Sections. The transformation 
algorithm from Section 4, which compiles an ALP to CHRs is replaced by a gen- 
eral abductive algorithm that interprets the ALP directly. Of course, an inter- 
preter is slower than a compiled procedure. However, besides providing a clarifi- 
cation of our approach, the general abductive procedure can also be used as an 
intermediate step for proving the soundness and completeness of our partial-order 
planning algorithm for dependent fluents. (The implementation from Section 4.1 
above is more efficient due to its direct encoding in CHR, as opposed to using 
CHR just for interpreting positive and negative goals, as below. A number of 
problem and domain dependent decisions, such as declaring certain predicates to 
be "backward", also influence the efficiency of the implementation from Section 
4.1. Let us stress the fact that these decisions are entirely domain and problem 
dependent and that they are not a drawback of our general mechanism.) 



5.1 Open Predicates 

In Logic Programming, normal predicates are closed: their definition is assumed 
to be complete (Clark completion). On the other hand, abducibles in Abductive 
Logic Programming (ALP) are completely open, i.e. they can have any extension 
consistent with the integrity constraints. To formally deal with forward propaga- 
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tion rules in an abductive framework, we need to allow a generalization of ab- 
ducibles, namely (partially) open predicates. 

Unlike abducibles, open predicates can have definitions p Body, but these are not 
considered to be complete, since during the problem solving process we can add (by 
forward propagation) new abductive instances of p to these definitions. The definition 
of an open predicate therefore only partially constrains its extension. 

In our CHR embedding of the abductive procedure we shall use two types of con- 
straints, p and Cp, for each open predicate p. While Cp represent facts explicitly 
propagated (abduced), p refers to the current closure of the predicate p (i.e. the ex- 
plicit definition of p together with the explicitly abduced literals Cp). Thus, informally 
we have p = def(p) v Cp. 

While propagating Cp amounts to simply assuming p to hold (abduction), propa- 
gating p amounts to trying to prove p either by using its definition def(p), or by reusing 
an already abduced fact Cp.[^This distinction ensures that our CHR embedding con- 
forms to the usual ‘propertyhood view’ on integrity constraints; 

Definition M(A) is a generalized stable model of the abductive logic program (P,A,I) 
for the abductive explanation A c A iff 

(1) M(A) is a stable model of P uA, and (2) M(A) |= I. 

The distinction between propagating Cp and p respectively can be seen best in an 
example. When an action is applied, Chappens(A,T), we have to propagate its effects 
Eff as well as its preconditions Pre. But while propagating the effects simply involves 
the propagation of the constraint Cstarts(Eff,T), propagating the preconditions should 
entail posting the goal holds(Pre,T)|[]which amounts to trying to achieve Pre either by 
using its definition (and thus applying another action having Pre as an effect), or by 
reusing an already achieved fact. 

The use of open predicates allows mixing forward propagation of abduced predi- 
cates Cp with backward reasoning using the closures p. Forward propagation can be 
implemented using CHR propagation rules, while backward reasoning involves un- 
folding predicates with their definitions. The definition def(p, Body) of a predicate p is 
obtained by Clark completion of its ‘if definitions. For each such predicate we will 
have an unfolding rule (a CHR simplification rule) p <=> def(p, Body) | Body, but also a 
CHR simpagation rulef^for matching a goal p with an existing abduced fact Cp: 

CppCi) \ p(X2) oXi=X2 ; X1AX2, p(X2). 

This rule should be given a higher priority than the unfolding rule in order to 
avoid reachieving an already achieved goal. Combined with the forward propa- 
gation mechanism for Cp, it deals with the first non-minimality problem men- 



Abducibles (i.e. completely open predicates) p have no definition and are thus referred to as 
Cp. 

This is done by the CHR rule (10) which corresponds to the ALP integrity constraint (7). Note 
that while program clauses propagate Cp constraints, integrity constraints propagate goals p, in 
line with Lin and Reiter’s observation [6] that state (integrity) constraints are usually intimately 
tied with the qualification problem. 

The rule is more complicated in practice, due to implementation details. 
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tioned in Section 2. Note that, for completeness, we are leaving open the possi- 
bility of achieving p(X 2 ) using its definition or reusing other abduced facts. 

Our treatment of open predicates p = def(p) v Cp is slightly different than the usual 
method [8] of dealing with (partially) open predicates p by introducing a new predi- 
cate name p' (similar to our Cp) and adding the clause p ^ p' to the definition of p: 
{p^Def, p^p'}. (**) 

The difference is that whenever referring to p we are implicitly trying to prove p, 
either by using its definition def(p) or by reusing an already abduced fact Cp, but 
without allowing such a Cp to be abduced in order to prove p (whereas in (**) 
treating p p' as a program cZaM^e|^would allow p' to be abduced when trying 
to prove p). This is crucial for ensuring a correct distinction!^ between goals p 
and abducibles Cp mentioned above (otherwise we would treat the propagation of 
action preconditions incorrectly). Without making this distinction, we wouldn’t 
even be able to refer to the current closure of p. 



5.2 The Abductive Procedure 

The abductive procedure for open predicates given below is written using CHR 
rules.[^We assume that conjunction and disjunction in positive goals are 
dealt with implicitly (disjunction being treated by splitting). Integrity constraints 
<— G are represented as negative goals not(G).[^The order of rules does matter: 
the first rule matching a newly introduced constraint will be activated. If it is a 
simplification rule, the subsequent rules will not get the chance to be executed. 

In the rules below, we let p denote a predicate (possibly with variables). Multiple 
occurrences of p in a rule involves the unification of the corresponding literals. We 
also write p(X) (with X a tuple of variables) whenever we want to make the variables 
of p explicit. 

The abductive procedure presented above is change-oriented, since a change in the 
abducible Cp will trigger in rule [NEG-ABD] a matching negative goal, as in other 
abductive procedures. The main difference lies however in that Cp can be any open 
predicate, not just a completely open one. Such open predicates can be targets of for- 
ward propagation rules: Body =4> Cp. If these propagation rules represent the forward 
direction of some program rules p Body, then these program rules may be excluded 
when unfolding negative goals in [NEG-UNF]. There, def_(p. Body) returns the back- 
ward definitions of p, i.e. the Clark completion of the program rules for which no 
forward propagation rules have been written. For predicates p with no backward defi- 
nition (for example for abducibles), def_(p. Body) returns Body = fail and the rule 
[NEG-UNF] propagates not(fail) i.e. true. 



In fact, p <— p’ should be treated as an integrity constraint and not as a program clause. 

This distinction is essential only for partially open predicates and not for completely open 
predicates (abducibles). 

For lack of space, we omit the treatment of built-in constraints. 

hot’ is here a CHR constraint and should not be confused with the negation as failure operator 
used in logic programming. 
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Positive Goals 

[POS-ABD] Cp(Xi)\ p(X 2 ) O Xi=X 2 ; ~(Xi=X 2 ), p(X 2 ). 
[POS-UNF] p <=> def(p, Body) | Body. 



Negative Goals 

|NEG-T,F] not( true ) <=> fail. 

not( fail ) O true. 

[NEG-UNF] not( p, G ) => def_(p,Body) I not( Body, G ). 
[NEG-ABD] Cp(Xi) \ not( p(X2), G ) <=> 

not( Xi=^X2, G ), not( p(X2), ~(Xi=3X2), G ) 



[NEG-DISJ] not( (pi; p 2 ), G ) not( pi, G ), not( p 2 , G ). 

[NEG-INACI] Cp(Xi) \ not( Gi, not p(X 2 ), G 2 ) <=> noVvars(X 2 ) I 
Xi=X 2 ; ~(Xi=X 2 ), not( Gi,not p(X 2 ), G 2 ) 

[NEG- SPLIT] not( not p(X), G ) <=> noVvars(X) | p(X) ; not( p(X) ), not( G ). 

The main improvements of this abductive procedure consist in solving the problems 
of reachieving already achieved goals (mentioned in Section 2): 

• in the case of positive goals by forward propagation 

• in the case of negative literals in negative goals by inactivating the negative goals 
(using [NEG-INACT], whenever possible) before splitting them (using [NEG- 
SPLIT]). This also relies on forward propagation. 

Negative goals can contain both universal (V) and existential (3) variables (the latter 
correspond to anonymous constants occurring in the constraint store). For variable 
tuples Xi and X 2 we denote by Xi=jX 2 the set of equations obtained after eliminating 
(unifying away) the V variables. 0The noVvars(X) condition in the guard of [NEG- 
SPLIT] succeeds whenever the variable tuple X contains no universal variables. Its 
role is to avoid floundering. not( p(X) ) in the second disjunct of the consequent of 
[NEG-SPLIT] implements a form of semantic splitting. 

We have assumed conjunctions in the above algorithm to be ordered by a selection 
function. The selection function in a negative goal would typically prefer completely 
open predicates to other predicates and leave negative literals at the end. To avoid 
floundering, it would also try to choose only negative literals with no universal vari- 
ables. 

Our abductive procedure can be easily proved to be sound and complete. (The for- 
mal proof - which cannot be given here for lack of space - extends the standard proof 
for SLDNFA [2].) 



For solving the planning problem with our general abductive procedure, we can 
simply use the general abductive logic program given by (3)-(8) together with the for- 
ward propagation rule (9) for starts. (Thus the rule (6) is "forward", the other ones, 
namely (3)-(5), being labeled "backward".) 



For example, if Xi = [Y,Z,Z], X2=[A,B,C] with Y,Z V variables and A,B,C 3 variables, Xi=3X2 is 
the set of equations {B=C} obtained after getting rid of Y and Z by Y=A, Z=B. We consider 
both cases Xi=3X2 and ~(Xi=3X2) in order to leave open the possibility that B^^C. 
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6 Concluding Remarks 

Several related works, such as [9], use forward propagation for incrementally check- 
ing the consistency of deductive databases. Although they use forward propagation, 
the Kowalski-Sadri [10] and related algorithms are not appropriate for our purposes, 
since they only check the consistency of an update. The intermediate forwardly propa- 
gated facts are neither retained, nor reused after the check. These algorithms are there- 
fore of no help to us for avoiding reachieving in a different way an already achieved 
goal. 

On the other hand, the Suspended Logic Programs (SLPs - similar with Fung’s IFF 
procedure) of [5], also allow a combination of backward goal-oriented reasoning with 
forward propagation of necessary properties. SLPs are very similar to CHRs in that 
insufficiently instantiated goals (that do not match any head of their iff definitions) are 
suspended. Thus, suspension is used as a mechanism for avoiding the combinatorial 
explosions that would be entailed by unfolding insufficiently instantiated goals. In- 
stead of unfolding them, forward rule|^ are used to propagate the properties of such 
suspended goals in the hope of discovering any potential inconsistencies or for further 
instantiating the goal variables and thus allowing their unfolding. 

Unfortunately, the propagated properties have to be unfolded as well, which may 
lead to a blow-up of the computation, unless special care is given to suspension con- 
troll^ In our terminology, such SLP propagation rules propagate goals of the form p 
(which are subject to unfolding, leading to potential blow-ups), while we only propa- 
gate forward constraints of the form Cp (which are not subject to unfolding). Thus, 
SLPs represent a more general architecture (very much like CHRs), while we are 
developing a more specific abductive procedure, being more concerned with dealing 
with the non-minimalities in existing abductive procedures. 

As far as we know, our planner is the first sound and complete partial-order planner 
able to deal with dependent fluents and non-ground plans. The prototype CHR imple- 
mentation has been successfully tested as query planner of the mediator in the frame- 
work of system integration. 
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Abstract. Coordinating the motion of multiple mobile robots is one of the funda- 
mental problems in robotics. The predominant algorithms for coordinating teams 
of robots are decoupled and prioritized, thereby avoiding combinatorially hard 
planning problems typically faced by centralized approaches. While these meth- 
ods are very efficient, they have two major drawbacks. First, they are incomplete, 
i.e. they sometimes fail to find a solution even if one exists, and second, the re- 
sulting solutions are often not optimal. In this paper we present a method for 
finding and optimizing priority schemes for such prioritized and decoupled plan- 
ning techniques. Existing approaches apply a single priority scheme which makes 
them overly prone to failure in cases where valid solutions exist. By searching 
in the space of priorization schemes, our approach overcomes this limitation. It 
performs a randomized search with hill-climbing to find solutions and to minimize 
the overall path length. To focus the search, our algorithm is guided by constraints 
generated from the task specification. To illustrate the appropriateness of this ap- 
proach, this paper discusses experimental results obtained with real robots and 
through systematic robot simulation. The experimental results illustrate the supe- 
rior performance of our approach, both in terms of efficiency of robot motion and 
in the ability to find valid plans. 



1 Introduction 

Path planning is one of the fundamental problems in mobile robotics. As mentioned by 
Latombe mg, the capability of effectively planning its motions is “eminently necessary 
since, by definition, a robot accomplishes tasks by moving in the real world.” 

In this paper we consider the problem of motion planning for multiple mobile robots. 
In particular, we are interested in planning paths for multiple robots operating in a 
single, shared environment, where physical limitations impose restrictions among the 
paths of the various robots. In such multi-robot problems, undesirable situations include 
congestions or deadlocks, which may prevent robots from reaching their goal locations. 
Since the size of the joint state space of the robots grows exponentially in the number 
of robots, planning paths for teams of mobile robots is significantly harder than the path 
planning problem for single robot systems. Therefore, existing approaches for single 
robot systems cannot directly be transferred to multi-robot systems. 

The approaches for multi-robot path planning can roughly be divided into two ma- 
jor categories lEl: centralized and decoupled. In the centralized approach I3II9I the 
configuration spaces of the individual robots are combined into one composite configu- 
ration space which is then searched for a path for the whole composite system. Because 
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Fig. 1. Situation in which no solution can be found if robot 3 has higher priority than robot 1 . 



the size of the joint configuration space grows exponentially in the number of robots, 
this approach suffers intrinsic scaling limitations. The major alternative are decoupled 
approaches 1711 71 131512 111 1811 . Decoupled approaches compute separate paths for the 
individual robots. Subsequently, they apply heuristics for resolving conflicts between 
different robots (e.g., two robots attempt to occupy the same location at the same time). 
To deal with the still enormous search space, it is common practice to assign priorities to 
the individual robots I7I5I21I1RI| . Planning and re-planning is performed in accordance 
with these priorities. Priority schemes provide an effective mechanism for resolving 
conflicts that is computationally extremely efficient. 

However, the priority scheme has a strong influence on whether a solution can be 
found and on how long the resulting paths are. To illustrate this, let us consider two 
examples. Figure Q] shows a situation in which no solution can be found robot 3 has 
a higher priority than robot 1 . Since the path of robot 3 is planned without considering 
robot 1, it enters the corridor containing its target location (marked G3) before robot 1 
has left this corridor. Since the corridors are too narrow to allow two robots to pass by, 
robot 3 blocks the way of robot 1 so that it cannot reach its target point Gl. However, 
if we change the priorities and plan the trajectory of robot 1 before that of robot 3, then 
robot 3 considers the trajectory of robot 1 during path planning and thus will wait in the 
hallway until robot 1 has left the corridor. Another example is shown in Figure Q(left). 
If we start with robot 1 then we have to choose a large detour for robot 2 (see FigureO 
center). This is because robot 1 blocks the corridor. However, if the path of robot 2 is 
planned first, then we can obtain a much more efficient solution (see FigureQ right). 
These two examples illustrate that the priority scheme has a serious influence on whether 
a solution can be found and on how long the resulting paths are. Moreover, it suggests 
that no single prioritization scheme will be sufficient for all possible multi-robot motion 
problems. 

In this paper, we present a technique that searches in the space of all priority schemes 
while solving hard multi-robot planning problems. Our approach performs a randomized 
hill-climbing search in the space of possible priority schemes. Since each change of a 
scheme requires the computation of the paths for many of the robots, it is important to 
focus the search. Our method achieves this by exploiting constraints between the different 
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Fig. 2. Independently planned optimal paths for two robots (left), sub-optimal solution if robot 1 
has higher priority (center), and solution resulting if the path for robot 2 is planned first (right) . 



robots which are derived from the task specification. This has two serious advantages. 
First, it reduces the time required to find a solution, and second, it increases the number 
of problems for which a solution can be found in a given amount of time. Additionally, 
our algorithm is ahle to reduce the overall path length. It has anytime characteristics |l'2'2]. 
which means that the quality of the solution depends on the available computation time; 
however, a solution may be available at any point in time. 

Our approach has been successfully applied to physical mobile robots. These results 
are complemented by extensive simulations, to characterize the relation between the 
planning performance and various problem parameters. In all experiments, we found 
that our approach produces highly efficient motion plans even for very large teams of 
robots, for different environments, and using two different decoupled path planning 
techniques. 

The paper is organized as follows. After discussing related work in the following sec- 
tion, we introduce two decoupled path planning techniques that will he used throughout 
this paper. SectionEI describes our algorithm for searching for priority schemes during 
planning. Finally, in Section El we present systematic experimental results illustrating 
the capabilities of our approach. The paper is concluded in Sectional 

2 Related Work 

The problem of coordinating multiple mobile robots has received considerable attention 
in the robotics literature. As already mentioned above, the techniques for multi-rohot path 
planning can roughly he divided into the centralized and the decoupled approaches 1121 ■ 

Centralized methods consider the composite configuration space of all robots and 
search for a solution in the whole composite system. While these approaches (at least 
theoretically) are able to find the optimal solution to any planning problem for which a 
solution exists, their time complexity is exponential in the dimension of the composite 
configuration space. In practice, one is therefore forced to use heuristics for the explo- 
ration of the huge joint state space. Many methods use potential fields Il2i:'il2()l to guide 
the search. These techniques apply different approaches to deal with the problem of lo- 
cal minima in the potential function. Other methods restrict the motions of the robots to 
reduce the size of the search space. For example, I‘)I1‘)I1 111 only consider trajectories that 
lie on independent road-maps. The coordination is achieved by searching the Cartesian 
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product of the separate road-maps. Nevertheless, centralized approaches scale poorly to 
large numbers of robots. 

Decoupled planners, in contrast, determine the paths of the individual robots inde- 
pendently and then employ different strategies to resolve possible conflicts. According 
to that, decoupled techniques are incomplete, i.e., they may fail to find a solution even 
if there is one. A popular decoupled approach is planning in the configuration time- 
space m, which can be constructed for each robot given the positions and orientations 
of all other robots at every point in time. Techniques of this type assign priorities to the 
individual robots and compute the paths of the robots based on the order implied by these 
priorities. The method presented in CD uses a fixed order and applies potential field 
techniques in the configuration time-space to avoid collisions. The approach described 
in fS] also uses a fixed priority scheme and chooses random detours for the robots with 
lower priority. 

Another approach to decoupled planning is the path coordination method which 
was first introduced in El. The key idea of this method is to keep the robots on their 
individual paths and let the robots stop, move forward, or even move backward on their 
trajectories in order to avoid collisions (see also Il6l4ll '). To reduce the complexity in the 
case of huge teams of robots, Leroy and colleagues |fni| recently presented a technique to 
separate the overall coordination problem into sub-problems. Their approach, however, 
assumes that the overall problem can be divided into very small sub-problems. As various 
examples described below demonstrate, this assumption may not be justified in certain 
situations. 

Unfortunately the problem of finding the optimal schedule is NP-hard for most of 
the decoupled approaches. To see, we notice that the NP-hard Job-Shop Scheduling 
problem with the goal to minimize maximum completion time 1141121 can be regarded 
as a special instance of the path coordination method. The decoupled and prioritized 
methods described above leave open how to assign the priorities to the individual robots. 
In the past, different techniques for selecting priorities have been used. For example, 
in I5’i heuristic techniques are described that assign higher priority to robots which can 
move on a straight line from the starting point to their target location. In Q all possible 
priority assignments are considered. Due to its (exponential) complexity this approach 
has only been applied to groups of up to three robots. 

In this paper we present an approach to optimize priority schemes for arbitrary 
decoupled path planning methods. We perform a randomized hill-climbing search in 
the space of priority schemes. Thereby, we interleave the search for an optimal priority 
scheme with the planning of the paths of the robots. To guide the search, our algorithm 
exploits constraints between the robots that are extracted from the task description. As 
a result, our approach seriously reduces the time needed to find a solution to the path 
planning problem. Once a solution has been found, our algorithm is able to optimize the 
priority scheme in order to minimize the overall path length. 



3 Prioritized A*-Based Path Planning and Path Coordination 

The basic algorithm to compute optimal paths for single robots, which will be used 
throughout this paper, is a variant of the popular A* search procedure tT^ . To represent 
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the environment of the robots we apply occupancy grids liT51 which separate the envi- 
ronment into a grid of equally spaced cells and store in each cell {x, y) the probability 
P{ocCx^y) that it is occupied by a static object. In this section we also present the key 
ideas of decoupled prioritized path planning and describe how the A* procedure can be 
utilized to plan the motions of teams of robots. 

3.1 A* -Based Path Planning 

Our system applies the A* procedure to compute the cost-optimal paths for the individual 
robots, in the remainder denoted as the independently planned optimal paths for the 
individual robots. A* addresses the problem of finding a shortest path from an initial 
state to a goal state in a graph. To search efficiently, the A* procedure takes into account 
the accumulated cost of reaching a certain location (a;, y) from the starting position, and 
an estimate of the cost of reaching the target location (x*,y*) from {x, y). By doing so, 
A* tends to focus its search in parts of the state space most relevant to the problem of 
finding a shortesf path. This property, which makes A* an efficient search algorithm, has 
given A* an enormous popularity in the robotics community. However, A* also requires 
a discrete search graph, whereas robot configuration spaces are continuous. In our case 
we assume that the environment is readily represented by a discrete occupancy grid 
map — which is common in the mobile robotics literature. 

The cost for traversing a cell (a;, y) is proportional to its occupancy probability 
P{ocCx,y)- Furthermore, the estimated cost for reaching the target location is approx- 
imated by c • ||(a;, y) — {x* , y*)\\ where c > 0 is chosen as the minimum occupancy 
probability P{ocCx,y) in the map and ||(a:,y) — {x*,y*)\ \ is the straight-line distance 
between {x, y) and (a;*, y*). Since this heuristic is admissible (see [lij). A* determines 
the cost-optimal path from the starting position to the target location. 

3.2 Decoupled Path Planning for Teams of Robots 

A* can easily be extended to the problem of decoupled and prioritized path planning. 
Recall that in the multi-robot path planning problem, many robots simultaneously seek 
to traverse an environment. If the robots could move freely regardless of other robot’s 
positions, the problem could easily be decoupled into many local path planning problem, 
in which each robot applied A* to determine its optimal path. However, the impossibility 
for robots to occupy the same location at the same point in time introduces non-trivial 
restrictions that have to be incorporated into the individual robot paths. 

A common approach is the following. In a first path planning step, each robot com- 
putes its optimal path using A*, without any consideration of the paths of the other 
robots. Clearly, the resulting paths might not be admissible since they lead to collisions, 
if executed. Thus, in a second planning step, each robot checks for possible conflicts 
with all other robots. Conflicts between robots are then resolved by introducing a priority 
scheme. A priority scheme determines the order in which the paths for the robots are 
re-planned. The path of a robot is then planned in its configuration time-space computed 
based on the map of the environment and the paths of the robots with higher priority. 

Please note, the A* search can also be used to plan the motions of the robots in 
the configuration time-space. As in the standard approach described above, the cost 
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of traversing a location (a;, y) at time t is determined by the occupancy probability 
P{ocCx,y)- To incorporate the restrictions imposed by the paths of the other robots, 
however, we do not allow a robot to enter a cell that is occupied by a robot with higher 
priority at time t. In addition to the general A* -based planning in the configuration time- 
space we consider a second and restricted version of this approach denoted as the path 
coordination technique O- It differs from the general approach in that it only explores 
a subset of the configuration time-space given by those states which lie on the initially 
optimal paths for the individual robots. The path coordination technique thus forces the 
robots to stay on their initial trajectories. The overall complexity of both approaches is 
0{n ■ m ■ log(m)) where n is the number of robots and m is the maximum number of 
states expanded by A* during planning in the configuration time-space (i.e. the maximum 
length of the OPEN-list). Due to the restriction during the search, the path coordination 
method is more efficient than the general A* search. Its major disadvantage, however, 
lies in the fact that it fails more often. 

As already discussed above, the introduction of a priority scheme for the decoupled 
path planning leads to serious reduction of the overall complexity. Whereas there are 
schemes leading to a viable solution with collision-free paths, it is easy to see that 
there are schemes for which no solution can be found. In addition to the fact, that the 
order in which the robots may plan their paths has a profound impact on the ability 
of finding a solution, even the quality of the solution depends heavily on the priority 
scheme. Examples of such situations were already discussed in the introduction to this 
paper. Unfortunately, the problem of finding the optimal priority scheme, is a non-trivial 
matter. More specifically, the NP-hard Job-Shop Scheduling problem with the goal to 
minimize maximum completion time inuni can be regarded as an instance of the 
path coordination method. Therefore, we have to be content with possibly sub-optimal 
planning orders. 



4 Finding and Optimizing Solvable Priority Schemes 

This section describes our approach to searching in the space of priority schemes during 
decoupled path planning. As the examples given in Figures Q]and|3illustrate, the order 
in which the paths are planned has a significant influence on whether a solution can be 
found and on how long the resulting paths are. This raises the question of how to find 
a priority scheme for which the decoupled approach does not fail and how to find fhe 
order of the robots leading to the shortest paths. 

4.1 The Randomized Search Technique 

Our algorithm for finding eligible priority schemes is a randomized search technique, 
similar to those reported in [131 . More specifically, our approach performs a randomized 
and hill-climbing search in order to optimize the planning order for decoupled and 
prioritized path planning techniques. Our approach starts with an arbitrary initial priority 
scheme U and randomly exchanges the priorities of two robots in this scheme. If the 
new order II' results in a solution with shorter paths than the best one found so far, 
we continue with this new order. Since hill-climbing approaches like this frequently get 
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Stuck in local minima, we perform random restarts with different initial orders of the 
robots. Thus, our approach interleaves the search for collision-free paths with the search 
for a solvable priority scheme. 



4.2 Exploiting Constraints to Focus the Search 

Whereas the plain randomized search technique produces good results, it has the major 
disadvantage that often a lot of iterations are necessary to come up with a solution. For 
example, we found that for ten robots in the environment shown in Figure [^more than 
20 iterations on average were necessary to find a solvable priority scheme. In this section 
we therefore present a technique to focus the search that tends to reduce the search time 
significantly. Our approach can be motivated through the situation depicted in Figure[I] 
In this situation, it impossible to find a path for robot 1 if the path of robot 3 is planned 
first, because the goal location of robot 3 lies on the optimal path for robot 1 . The key 
idea of our approach is to introduce a constraint pi > pj between the priorities of two 
robots i and j, whenever the goal position of robot j lies on the optimal path of robot 
i. In our example we thus obtain the constraint pi > p^ between the robots I and 3. 
Additionally, we get the constraint p 2 > Pi, since the goal location of robot 1 lies too 
close to the trajectory of robot 2. 

Although the satisfaction of the constraints by a certain priority scheme does not 
guarantee that valid paths can be found, orders satisfying the constraints more often 
have a solution than priority schemes violating constraints. Unfortunately, depending 
on the environment and the number of the robots, it is possible that there is no order 
satisfying all constraints. In such a case the constraints produce a cyclic dependency. 
The key idea of our approach is to initially reorder only those robots that are involved in 
such a cycle in the constraint graph. Thus, we separate all robots into two sets. The first 
group i?i contains all robots that, according to the constraints, do not lie on a cycle and 
have a higher priority than the robot with highest priority which lies on a cycle. This set 
of robots is ordered according to the constraints and this order is not changed during the 
search. The second set, denoted as i ?2 contains all other robots. 

As an example. Figure 01 (left) shows a simulated situation with ten robots. Whereas 
the starting positions are marked by Sq, ... ,Sg the corresponding goal positions are 
marked by Go, . . . , Gg. The independently planned optimal trajectories are indicated 
by solid lines. Given these paths we obtain the constraints depicted in Figure 01 (right). 
According to the constraints, six robots belong to the group of robots whose order (at 
least in the beginning) remains unchanged during the search process. The robots in their 
order of priorities are 3, 6, 7, 2, 4, 9. 

Initially, our algorithm only changes the order of the robots in the second group. 
After k iterations, we include all robots in the search for a priority scheme. In extensive 
experimental results we figured out that this approach leads to better results with respect 
to the overall path length, especially for large numbers of iterations. The complete 
algorithm is listed in Tabled 

If we apply this algorithm to the example shown in Figure 0 (left) under the con- 
straints shown in Figure 0 the system quickly finds a solution. One typical result is the 
the order 0, 1, 5, and 8, for those robots that generate a cycle in the constraint graph. 
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Table 1. The algorithm to optimize priority schemes. 



count := 0 

FOR tries := 1 TO maxTries BEGIN 

IF count > k // extensive search after k iterations 
select random order 71 

ELSE 

select order 77 given fixed order for Ri 
and random order for R 2 
IF (tries = 1) 

77* := 77 

FOR flips := 1 TO maxFlips BEGIN 

IF count > k // extensive search after k iterations 
choose random i, j with i<j 

ELSE 

choose random i, j with i<j and i,j£ R 2 
n' := swap(i, j, 77) 
count := count+1 

IF moveCosts (770 < moveCosts (77) 

77 — 77' 

END FOR 

IF moveCosts (77) < moveCosts (77*) 

77* := 77 

END FOR 
RETURN 77* 



The corresponding collision-free paths for all robots are shown in FigureOl This demon- 
strates, that the constraints drastically reduce the search space and still allow the system 
to quickly find solvable priority schemes. 



5 Experimental Results 



Our approach has been tested thoroughly on real robots and in extensive simulation runs. 
The two key questions addressed in our experiments were: (1) Solvability: Does our ap- 
proach succeed more frequently in finding valid multi-robot paths than approaches with 
fixed prioritization? (2) Optimality: If our approach succeeds, does it generate more effi- 
cient plans? All experiments were carried out using different environments. To evaluate 
the general applicability, we applied our method to the two decoupled and prioritized 
path planning techniques described above. The current implementation is highly effi- 
cient. It requires less than 0. 1 seconds on a 1 000 MHz Pentium III to plan a collision-free 
path for one robot in all environments described below. The whole optimization for 10 
robots with 10 restarts and 10 iterations per restart requires approximately one minute. 
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Fig. 3. Independently planned paths for ten robots (left) and the paths resulting after priority 
optimization (right). 




Fig. 4. Constraintgraph generated according to the paths shown in Figure|2](left). 



5.1 Simulation Experiments 

To elucidate the scaling properties of our approach to larger number of robots, we per- 
formed extensive simulation experiments. In particular, we were interested in character- 
izing the dependence between the performance of our system on various components of 
our approach. In our experiments, we analyzed the number of planning problems that 
can be solved using our strategy, the speed-up obtained by exploiting the constraints, and 
the reduction of the overall path length. In all experiments, we found that our approach 
produces highly efficient motion plans even for very large teams of robots, for different 
environments, and regardless of the specific baseline path planning technique (e.g.. A*). 

Solved Planning Problems. This first set of experiments was designed to characterize 
the effect of our search scheme on the overall number of failures. For each number of 
robots considered, we performed 100 experiments. In each experiment we randomly 
chose the starting and target locations of the robots. We applied four different strategies 
to find solvable priority schemes: 

1. A single randomly chosen order for the robots. 

2. A single order which satisfies the constraints for the robots in Ri and consists of a 
randomly chosen order for the robots in R 2 . 

3. Unconstrained randomized search starting with a random order and without consid- 
ering the constraints. 
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Fig. 5. Cyclic corridor environment used for the simulation runs . 




number of robots 




Fig. 6. Solved planning problems for different strategies using A* -based planning in the configura- 
tion time-space in the cyclic corridor environment depicted in Figure0(left) and the corresponding 
results obtained in the noncyclic corridor environment shown in Figure0(right). 



4. Constrained randomized search starting with an order computed in the same way as 
strategy 2). 

All four strategies can be cast as special cases of our algorithm. In the first two strategies 
the corresponding values for maxTries and maxFlips are 1. For the first strategy the 
value of the threshold k is 0. The strategies 3 and 4 only differ in the value of the threshold 
k. Whereas the unconstrained search is obtained by setting fc = 0, the constrained search 
corresponds to a value of k = oo. 

Please note that in this experiment we chose a small number of iterations for the last 
two strategies in order to assess the advantages of the constrained search under serious 
time constraints. Particularly, we chose a value of 3 for the parameters maxFlips and 
maxTries. Obviously, the larger the number of iterations, the higher is the probability 
that a solution can be found by an arbitrary randomized search. However, larger numbers 
of iterations drastically increase the computation time. For each technique, we performed 
A* -based planning in the configuration time- space and counted the number of solved 
planning problems. 

Figure El (left) summarizes the results we obtained for the cyclic corridor environ- 
ment depicted in FigureEl The horizontal axis represents the number of robots, and the 
vertical axis depicts the percentage of solved path planning problems. As this result illus- 
trates, our constrained search technique succeeds more often than any of the alternative 
strategies. It is interesting to note that the second strategy, which exploits the constraints 
but considers only one scheme in each experiment, shows a similar performance than the 
unconstrained randomized search. To complement these results, we performed a similar 
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number of robots 



Fig. 7. Solved planning problems for all four strategies using the path coordination method in the 
noncyclic environment depicted in Figure[21(left). 



series of experiments for the noncyclic corridor environment depicted in Figure 0 The 
results are shown in Figure0(right). Again, our constrained-based search outperforms all 
other strategies. All these and the following results are significant on the 95% confidence 
level. 

To investigate the performance using a different baseline path planning algorithm, 
we applied all four strategies using the path coordination method instead of plain A* . 
We used a variant of the environment depicted in Figure 01 with five corridors on both 
sides. Since the path coordination method restricts the robots to stay on their indepen- 
dently planned optimal trajectories, the number of unsolvable problems is much higher 
compared to the A* -based planning in the configuration time-space. As can be seen 
from FigureQ our constrained search leads to a much higher success rate that actually 
increases with the number of robots involved. 



Speed-up Obtained by Exploiting the Constraints. In this section, we are interested in 
one particular aspect of our approach, namely the ability to guide the search in the space 
of all priority schemes. More precisely, we pose the question how much the computation 
time necessary to find a solution can be reduced by constraining the search. 

For the next set of experiments we increased the values of maxFlips and maxTries 
to 10 and evaluated in which iteration the first solution was found if the planning problem 
could be solved. Figure[BI(left) plots the results obtained for different number of robots 
in the cyclic corridor environment and Figure[BI (right) shows the same evaluation for the 
noncyclic environment. As can be seen, for both environments the unconstrained search 
needs significantly more iterations to generate a solution. 

As these experiments suggest, the advantages of our constrained search is two-fold. 
On one hand, it requires fewer iterations than the unconstrained counter-part. On the 
other hand, it requires less computation, since the search is restricted to a subset of the 
robots, which reduces the number of paths that are generated in the search. 

Influence on the Overall Path Length. The previous experiments investigated the 
number of cases in which a solution can be found, as a function of the algorithm used for 
path planning. In this section, we will be interested in plan efficiency, that is, the overall 
plan execution time. 
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Fig. 8. Iteration in which the first solution was found if the planning problem could be solved 
for the cyclic corridor environment (left) and the corresponding results obtained int the noncyclic 
corridor environment (right). 




Fig. 9. Independently planned optimal paths for 30 robots (left) and the resulting paths after priority 
optimization (right). 



To show that our optimization technique is not limited to typical corridor environ- 
ments, Figure 0(left) shows the independently planned optimal paths for a team of 30 
robots in an unstructured environment. By optimizing these paths over 100 iterations, 
we obtain the solution illustrated in Figure (right). Figure^^(left) plots the evolution 
of the summed move costs of the best solution found so far over time. As can be seen 
from the figure, after 100 iterations the overall move costs are reduced by 15%. 

The final experiment in this section is designed to analyze the performance of our 
algorithm with respect to the overall path length. Since our algorithm in the beginning 
only considers a restricted set of priority schemes, and after k iterations explores the 
whole set of priority schemes, we are especially interested in how long the resulting 
paths are compared to the unconstrained search. We performed over 100 experiments 
in the cyclic corridor-environment and determined the average overall move costs at 
every iteration. The corresponding graphs are shown in Figure [El (right). This plot 
contains the average move costs for three different strategies at each iteration. The first 
data set was obtained for the constrained search which corresponds to k = oo. Using 
this strategy we reorder only those robots which lie on a cycle in the constraint graph. 
The data for the unconstrained search was obtained using fc = 0. In this case our 
algorithm chooses arbitrary priority schemes regardless of the constraints which were 
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Fig. 10. Summed move costs plotted over time for the planning problem with 30 robots shown in 
Figure 0 (left) and summed move costs plotted over time averaged over 100 planning problems 
for 15 robots in the cyclic environment (right). 



extracted given the task specification. Finally, the third function labeled “combining both 
techniques” corresponds to the results obtained with our algorithm given k = 20. 

Since the constrained search, which is guided by our heuristics, focuses the search 
on the robots that pose the most serious restrictions to the other robots, it more quickly 
finds a solution and accordingly has more time to optimize it. Thus, in the beginning, the 
constrained search outperforms the unconstrained search. After 20 iterations, however, 
the situation completely changes. Because the unconstrained search can explore many 
more priority schemes, it more often finds better solutions than the constrained search. 
Thus, after 20 iterations, the unconstrained search leads to better results than the con- 
strained search. As can be seen from the figure, our approach combines the advantages 
of both methods. In the beginning, it applies the constraints to focus the search and 
to quickly find a first solution which is optimized subsequently. After 20 iterations it 
considers arbitrary priority schemes so that the resulting path length is reduced as in the 
unconstrained search. 

Accordingly, our randomized search that initially uses the constraints to focus the 
search for a viable solution and afterwards uses the unconstrained search to optimize 
this solution inherits the advantages of both techniques with respect to efficiency and 
the overall resulting path length. 

5.2 An Example with Two Real Robots 

Figure (center) illustrates a typical application example carried out in our office 
environment with our robots Albert and Ludwig. The robots are shown in Figure 
(left and right). In this example, we used the general A* procedure in the configuration 
time-space for local path planning. While Albert starts at the right end of the corridor of 
our lab and has to move to left end, Ludwig has to traverse the corridor in the opposite 
direction. Notice that no path for Albert can be found if the path of Ludwig is planned 
first, since Albert cannot reach its target point if Ludwig stays on its optimal trajectory. 
Because of that, the system alters the order of the two robots. Given the optimal path for 
Albert, our system plans a path for Ludwig which first leads it into a doorway in order to 
let Albert pass by. The resulting trajectories are shown in Figure[n](center). Notice that 
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Fig. 11. The mobile robots Albert (left) and Ludwig (right) and a real world application of A* - 
based planning in the configuration time-space where Ludwig moves away in order to let Albert 
pass by (center) . 



at some point, the robot Ludwig waits to let the robot Albert pass by. In comparison, no 
solution can be found in this situation if the path coordination 11 Sll technique is used. 

In various other tests operating our two robots in our narrow hallways, we frequently 
observed the emergence of solutions where robots sensibly coordinated their behavior, 
e.g., by waiting for each other. However, we also notice that with only two robots, these 
experiments do not evaluate the utility of our search algorithm in priority scheme space, 
since there exist only two such schemes. Unfortunately, we currently have only two 
physical robots available in our lab, so that the experiment could not be carried out with 
larger groups of robots. 

6 Conclusions 

This paper presented an approach to optimize priority schemes for arbitrary decoupled 
and prioritized path planning methods for groups of mobile robots. Our approach per- 
forms a randomized hill-climbing search in the space of priority schemes in order to find 
a solution and to minimize the overall path length. To guide the search, our approach 
exploits constraints extracted from the current task specification. 

The approach has been implemented and tested on real robots. In addition, exten- 
sive simulations were performed to complement the physical robot experiments. The 
experiments suggest that our technique significantly decreases the number of failures 
in which no solution can be found, compared to a range of alternative approaches. Ad- 
ditionally, our approach leads to a significant reduction of the overall path length. A 
further advantage of our method lies in its general applicability. Although we applied 
our optimization technique only to two different baseline path-planning techniques in 
this paper, it is not limited to these two techniques. Rather, it can be used to find and 
optimize paths generated with arbitrary prioritized path-planning techniques. 

Apart from the promising results presented in this paper, there are different aspects 
for future research. First, in the experiments carried out here, we assumed equal con- 
stant velocities for all robots. In practice, teams often are inhomogeneous and contain 
different types of robots with different average velocities which has to be taken into ac- 
count. Furthermore, the techniques considered here provide no means to react to possible 




92 



M. Bennewitz, W. Burgard, and S. Thrun 



deviations of the robots from their planned trajectories during the plan execution. For 
example, if one robot is delayed because unforeseen objects block its path, alternative 
plans for the robots might be more efficient. In such situations it would be important to 
have appropriate plan-revision techniques. Additionally, the delay of a single robot may 
result in a dead-lock during the plan execution. In this context, robot control systems 
require techniques to detect dead-locks while the robots are moving and to resolve such 
dead-locks appropriately. 
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Abstract. A possible worlds semantics is suggested for a broad class of 
nonmonotonic inference relations, including not only traditional skeptical 
ones, but also credulous and contraction inference. The semantics could 
be used to provide a canonical framework for studying and comparing 
different kinds of nonmonotonic inference. 



1 Introduction 

One of the important approaches to nonmonotonic reasoning consists in describ- 
ing associated nonmonotonic inference relations and their semantics. Thus, the 
so-called KLM theory nUITTl has suggested a semantic representation of prefer- 
ential inference relations in terms of possible worlds models in which the world- 
states are ordered by a preference relation. In this framework, a nonmonotonic 
inference rule A\^ B was defined as saying that B should hold in all preferred 
world-states satisfying A. 

The above mentioned preferential inference relations were designed to capture 
a skeptical approach to nonmonotonic reasoning, according to which, if there is 
a number of equally preferred alternatives, we infer only what is common to 
all of them. However, works in nonmonotonic reasoning have suggested also 
an alternative approach, usually called credulous or brave reasoning, in which 
each of the preferred alternatives is considered as an admissible solution to the 
nonmonotonic reasoning task. Many important reasoning problems in AI, such 
as diagnosis, abduction and explanation, are best seen as involving the search 
for particular preferred solutions. This idea is implicit also in the notion of an 
extension in default logic and its generalizations, as well as in similar constructs 
in modal nonmonotonic logics. 

It turns out that preferential KLM models from nm are not suitable for 
representing the above notion of credulous inference. In |2|, an axiomatization 
and semantic representation of credulous inference has been suggested based on 
the notion of an epistemic state that can be viewed as a generalization of KLM 
models (see below). Epistemic states, however, are not possible worlds models, 
since their admissible states are labeled with deductively closed theories that are 
in general not complete. 
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In addition to brave nonmonotonic reasoning, a well-known correspondence 
between nonmonotonic inference and belief revision (see, e.g., 0) suggests yet 
another, quite different kind of nonmonotonic inference relation that corresponds 
to a contraction operation in the theory of belief change. This kind of inference 
relations has been introduced in |5], and it has also received a semantic repre- 
sentation in the framework of epistemic states. 

In this report we will show that the above ‘non-standard’ kinds of nonmono- 
tonic inference can also be given a traditional, though slightly unusual, possible 
worlds semantics. In fact, the only change we must do to KLM models in order 
to obtain such semantics consists in considering the preference relation among 
worlds to be a weak partial order, in contrast to strict partial orders usually used 
for representing preferential inference. Using such weak orders, and especially the 
derived equivalence relation on worlds, will allow us to give an exact semantic 
description for the above kinds of inference in the framework of ordinary possible 
worlds models. 

The possibility of such a possible worlds representation suggests an alterna- 
tive, more standard, viewpoint on the above inference relations. In particular, it 
opens the possibility of using standard modal logic techniques for their study. 
Finally, it allows us to make quite transparent comparisons between different 
kinds of nonmonotonic inference. 



1.1 Epistemic States 



As a preparation, we describe in this section the notion of an epistemic state 
that has been used in m for representing all the above-mentioned kinds of 
nonmonotonic inference. Epistemic states stem from a quite common under- 
standing according to which nonmonotonic reasoning uses not only known facts, 
but also defaults, or expectations, we have about the worlc^. These defaults are 
used as auxiliary assumptions that allow us to ‘jump’ to useful conclusions and 
beliefs that are not logical consequences of the facts alone. Such conclusions are 
defeasible and can be retracted when further facts become known to the rea- 
soning agent. In addition, not all defaults are equally plausible or acceptable, 
and this creates, in turn, priorities and preferences among otherwise admissible 
combinations of defaults. 

Since accepted combinations of defaults are primarily used for making in- 
ferences, admissible sets of defaults can be safely replaced by their deductive 
closures. So, instead of a prioritized collection of admissible sets of defaults, we 
can consider a set of deductively closed theories ordered by a preference rela- 
tion. A slight generalization of this picture leads to the following notion of an 
epistemic state: 



^ We use here a ‘naive’ understanding of defanlts as propositions (as in 0 and M), 
in contrast to formalizations of defaults made in default logic and some theories of 
nonmonotonic inference. 
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Definition 1. An epistemic state E is a triple where S is a set of 

objects admissible belief states, -< is a preference relation on S, while I is a 
labeling function assigning each admissible state a deductively closed theory. 

If s A t holds for two admissible belief states from E, we will say that t is 
preferred to s. For the purposes of this report, the above preference relation can 
be safely taken to be a strict partial order. 

Formally, epistemic states are similar to preferential models of Makinson m 
and cumulative models from ji()P . However, we will use epistemic states in a 
somewhat different way. The main difference stems from the understanding that 
these states are epistemic] in other words, they normally do not involve objective 
facts and evidences, but only our beliefs, defaults and expectations. Accordingly, 
we are primarily interested not in (preferred) admissible states that satisfy a 
given evidence, but rather in admissible states that are compatible with the 
evidence. For example, Poole’s abductive framework from m is representable 
as an epistemic state formed by all consistent combinations of defaults; the 
preference order amounts in this case simply to set inclusion. Then prediction 
and explanation in Poole’s system is determined by maximal sets of defaults 
that are consistent with given facts. 

An admissible state s £ S will be said to support a proposition Aif A £ l{s), 
and consistent with A if -lA ^ l{s). The set of admissible states that do not 
support A will be denoted by ]A[. Clearly, the set of admissible states that are 
consistent with A will coincide with ]~'H[ (while ]A[ is, of course, the set of states 
consistent with ~'A). 

Let P be an arbitrary set of admissible belief states from S. An admissible 
state s £ P will be said to be preferred in P if there is no admissible state t in 
P such that s ^ t. A set P of admissible states will be called smooth (see |Tn| ) 
if, for any s £ P, either s is preferred in P or there exists a preferred state t in 
P such that s ^ t. Finally, an epistemic state E will be called smooth if any set 
of belief states of the form ]A[ is smooth. 

As can be seen, the above definition of smooth epistemic states is different 
from that given in [IDj, since we require smoothness for sets of belief states that 
are consistent with some proposition. We will presume in what follows that our 
epistemic states are smooth. Note, however, that the smoothness requirement 
is trivial for finite epistemic states in which the preference relation is a partial 
order. 



2 Skeptical Inference 

Since epistemic states represent relatively stable default beliefs, in order to em- 
ploy them in particular evidential situations, we should restrict our attention to 
admissible belief states that are consistent with the current facts, and choose pre- 
ferred among them. The latter are used to support the conclusions and assump- 

^ Note that labeling with a deductively closed theory is equivalent to labeling with a 
set of worlds, as in m- 
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tions we make about the situation at hand. Accordingly, all kinds of nonmono- 
tonic inference are based on a two-step selection procedure: given a proposition 
E representing current evidence, we consider admissible belief states that are 
consistent with E and choose preferred elements in this set. Differences among 
various kinds of nonmonotonic inference will arise only at this point, due to 
different use we will make of these preferred belief states. 

A skeptical inference with respect to an epistemic state is obtained if we 
decide that, given the set of preferred belief states consistent with the facts, we 
can reasonably infer only what is supported by each of these states. In other 
words, A will be a skeptical conclusion from the evidence if in a given epistemic 
state E if each preferred admissible belief set in E that is consistent with E, 
taken together with E itself, implies A. Or, in still other words. 

Definition 2. A is a skeptical consequence of E ( notation E\^ A) in an epis- 
temic state if E ^ A is supported by all preferred admissible states in \~'E[. 

A set of conditionals A\^ B that are valid in an epistemic state E in ac- 
cordance with the above definition will be called a skeptical inference relation 
determined by E. Accordingly, we will say that a set of conditionals forms a 
skeptical inference relation if it is determined by some epistemic state. 

The above definition constitutes a straightforward generalization of the cor- 
responding definition of prediction in Poole’s abductive abductive framework 
(see CD- It provides also a generalization of the notion of an expectation-based 
inference, given in 0. 

Historical note. The above epistemic definition of conditionals is actually very 
old. In fact, the ‘standard’ definition of nonmonotonic inference, given in ma, 
derives from the relatively modern possible worlds theory of conditionals de- 
veloped by Stalnaker and Lewis. The above definition, however, can be traced 
back to the era before the discovery of possible worlds, namely to Frank Ramsey 
and John S. Mill. In fact, this semantic definition can be seen as a particular 
variant of the Ramsey test for conditionals (see 0). Though less familiar, it has 
also been used in the so-called ‘premise-based’ semantics for counterfactuals pro- 
posed by Veltman and Kratzer EE]- The relation between the two approaches 
to analyzing conditionals has been studied already by David Lewis in m- 

It turns out (see 0) that the above semantic representation determines ex- 
actly preferential inference relations from m- Notice that preferential possible 
worlds models from nm constitute a special case of epistemic states; namely, 
they correspond to epistemic states in which admissible states are labelled with 
‘worlds’ (maximal consistent sets). In this respect, the completeness result proved 
in m says, in effect, that already such world-based epistemic states are sufficient 
for representing any preferential inference relation. 

We will establish below a general correspondence between epistemic states 
and possible worlds models that will explain, to some extent, the above results. 
But first we describe the two alternative notions of nonmonotonic inference, 
mentioned in the introduction. 
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3 Credulous Inference 

A credulous inference with respect to an epistemic state is defined by assuming 
that we can reasonably infer (or explain) conclusions that are supported by at 
least one preferred admissible state consistent with the facts. In other words, A 
will be a credulous conclusion from the evidence if in a given epistemic state 
E, if at least one preferred admissible belief set in E that is consistent with E, 
taken together with E itself, implies A. In still other words. 

Definition 3. A is a credulous consequence of E (notation E\^A) in an epis- 
temic state if E ^ A is supported by at least one preferred admissible state in 
]^E[. 

As before, a set of conditionals A\v B that are valid in an epistemic state E 
will be called a credulous inference relation determined by E. 

There have been a few attempts in the literature to investigate the properties 
of credulous inference, mainly with negative conclusions that such an inference 
does not satisfy practically all ‘respectable’ rules (see, e.g., 14151 b Thus, a distinc- 
tive feature of credulous reasoning is that it does not allow to conjoin different 
conclusions derivable from the same premises (because they might be grounded 
on different preferred solutions). In other words, credulous inference renders in- 
valid the rule And. 

(And) If A\^ B and A C, then A\^ B A C . 

An important role of the rule And in the classification of inference systems 
has been pointed out already by Gabbay in |S|. 

In |2|, an axiomatic characterization of credulous inference has been pre- 
sented based on the approach to conditionals developed by van Benthem in p. 
The axiomatization amounted roughly to the removal of the above postulate 
And from the characterization of rational inference relations. The resulting ax- 
iomatization has been shown to be complete with respect to the above semantic 
notion of credulous validity: 

Theorem 1. An inference relation is credulous if and only if it coincides with 
the set of conditionals that are credulously valid in some epistemic state. 

The above result will be used below also for establishing completeness of 
credulous inference with respect to certain possible worlds semantics. 

3.1 Permissive Inference 

We will briefly describe now yet another kind of non-skeptical inference, namely 
permissive inference relations. It is intimately connected with the so-called X- 
logics suggested by Siegel and Forget in nni. 

An inference relation is called permissive if it satisfies all the postulates of 
preferential (skeptical) inference except Cautious Monotony, which is weakened 
to the following rule: 
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(Conjunctive Cautious Monotony) 11 A\^ B t\ C , then A f\ B C . 

It turns out (see |2|) that permissive inference is an exact dual of a credulous 
inference under the following duality transformation that is familiar from the 
literature on conditional logics as a duality between ordinary conditionals and 
mzg/it-conditionals: 



(Dual) A\^^ B = A fs- ->B or f 

Thus, if we apply this transformation to a credulous inference relation, we 
will obtain a permissive inference relation, and vice versa. 

The above duality can be used for obtaining the properties of permissive in- 
ference by ‘dualizing’ corresponding properties of credulous inference. Thus, the 
semantic interpretation of credulous inference immediately gives us the following 
semantic characterization of permissive inference in epistemic states: 

Definition 4. A conditional A\^ B will be said to be permissively valid in an 
epistemic state E if any preferred admissible state consistent with A is consistent 
also with A A B. 

A most plausible reading of a permissive conditional appears to be “A is 
normally consistent with i?” , since it asserts that all normal situations consistent 
with A are such that B could be added to them without losing consistency. Since 
the above semantic description corresponds precisely to the interpretation of the 
relation that is dual to credulous entailment, we immediately obtain 

Theorem 2. An inference relation is permissive if and only if it eoincides with 
the set of eonditionals that are permissively valid in some epistemic state. 

4 Contraction Inference 

In the general correspondence between nonmonotonic inference and belief 
change, contraction inference corresponds to the basic operation of belief con- 
traction. This augments the idea that belief change and nonmonotonic inference 
are “two sides of the same coin” and extends it to belief contractions. 

Our earlier definition of skeptical inference with respect to epistemic states 
displays the latter as a composite notion. Namely, it says that A\^ B holds if 
and only if the implication A — )> i? is supported by all preferred belief states 
that do not contain -lA. This construction resembles quite closely the two-step 
construction of revision in the belief change theory. According to the latter, in 
order to revise a belief set with a new, possibly incompatible belief A, we should 
first contract -<A from the belief set, and then expand the result by adding A. 
This resemblance suggests that skeptical inference can be expressed using a more 
fundamental, or primitive, concept corresponding to the contraction operation 
in belief revision. This concept can be described as follows: 
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Definition 5 . B is a contraction consequence of A (notation A'^\B) in an 
epistemic state if B is supported by all preferred admissible states in ]A[. 

Thus, A^B holds if is a plausible belief in the absence of A. We will 
call an expression of the form A'^ B a, eontraetion eonditional or, in short, a 
eontractional. An informal reading of such contractionals will be “In the absence 
of A, normally B 

Remark 1. Rules of the form “In the absence of A, accept R” are actually a sim- 
plest kind of default rules that constitute the subject matter of Reiter’s default 
logic US]. The relationship between contractionals and Reiter’s default rules re- 
mains yet to be explored; presumably, it would be an important step on still a 
long way towards a future general theory of nonmonotonic reasoning. The sim- 
ilarity suggests, however, that a eontractional A^ B could be read as “ Unless 
A, i?” , with the only reservation that we do not accept the usual presupposition 
associated with the latter expression, namely that A is by itself an unexpected 
(abnormal) condition. 

Given the above definition of contraction inference, we can re-define now 
skeptical inference as follows: 

A\^ B = —lA ^ A — >■ R 

As can be seen, the above definition, coupled with the semantic definition of 
contraction inference, gives us exactly the definition of skeptical inference with 
respect to epistemic states. Accordingly, many properties of the latter can be 
analyzed already on the level of contraction inference relations. 

Similarly to conditionals, a set of contractionals that are valid in an epistemic 
state E in accordance with the above definition will be called a contraction in- 
ference relation determined by E. An axiomatic characterization of contraction 
inference relations and its completeness with respect to epistemic states have 
been given in jSj. The latter paper contains also a number of representation re- 
sults for extensions of the contraction relation depending on various constraints 
imposed on epistemic states. These results cover a broad range of possible belief 
contraction functions including traditional AGM contractions, as well as con- 
tractions that do not satisfy the recovery postulate. 

For completeness sake, we present below an axiomatic characterization of 
contraction inference. To begin with, general contraction relations are inference 
relations satisfying the following postulates: 

Tautology A t 

And If A fy R and A fy C, then A R A C. 

Right Weakening If R 1= C and Afy R, then Afy C. 

Extensionality If 1= A o C and A fy R, then C fy R. 
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Partial Antitony If A A B ^ A, then A A B A C ^ A. 

Cautious Antitony If A A B ^ A and B ^C, then A A B ^C. 
Distributivity If C and B ^C, then A A B ^ C. 

Cautious Monotony IfAAB^AAC, then B ^C. 

The above postulates are still insufficient for giving a complete description of 
contraction inference with respect to epistemic states. We need also an additional 
postulate that fixes the result of ‘impossible’ contractions, such as contractions of 
tautologies. A simplest possible way of doing this is a ‘classical’ one, according 
to which if we are forced to disbelieve logical tautologies, we are allowed to 
believe anythin^l. This stipulation is actually presupposed by the above semantic 
definition of contraction inference with respect to epistemic states: when we are 
saying that A^B is valid in an epistemic state if B holds in all preferred belief 
states that do not satisfy A, this implies, in particular, that if A holds in all 
admissible states, then A^B will be (trivially) valid, for any B. 

A general contraction relation will be called simple if it satisfies 

Simple Failure If A ~| A, then A ~| f . 

Then a slight modification of the completeness result proved in |2| will give 
us the following 

Theorem 3. A contraction inference relation is simple iff it coincides with a 
set of contractionals valid in some epistemic state. 

5 Reflexive Possible Worlds Models 

In this section we are going to show that all the above described kinds of non- 
monotonic inference can also be given a possible worlds semantics. More exactly, 
we will give a representation of these inference relations in terms of possible 
worlds models in which the accessibility (preference) relation is a pre- order. 

Definition 6. A reflexive possible worlds model is a triple W = (IT, I, =4), where 
W is a set of states, I is a labeling function assigning each z G IT a maximal 
deductively closed theory (a ‘world’), and =4 is a pre-order on IT. 

Given a pre-order ^ (i.e., a reflexive and transitive relation), we can imme- 
diately define the corresponding strict partial order in a well-known way: 

s ^ t = s ^ t and t ^ s 

Thus, reflexive models include, in a sense, preferential KLM models. This 
allows us to extend the terminology adopted for the latter to reflexive models. In 

® In |H), a different, ‘conservative’ stipnlation has been used, since it corresponded 
more closely to the behavior of contraction operations in common theories of belief 
change. 
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particular, we will continue to use the notions of a preferred state and smoothness 
as applied to the above (defined) relation 

Weak partial order determines, however, a richer structure on possible worlds 
than a strict one. In particular, it allows us to express the idea that two states 
can be equally preferred: 



s ^ t = s ^ t and t ^ s 

This derived equivalence on states will play an essential role in representing 
non-standard inference relations, given below. 



5.1 General Transformation 

A general correspondence between epistemic states and reflexive possible worlds 
models will be established using a certain uniform transformation of epistemic 
states into their corresponding reflexive models. 

Given an epistemic state E = {S,l, a), we define the corresponding reflexive 
possible worlds model We = (W, /q, as follows: 

— W is a set of all world-state pairs (a, s), where s G S, and a is a world 
(maximal consistent set) such that l{s) C a. 

— The preference relation ^ on IT is defined as follows: 

(a, s) ^ (/?, t) iff s A t or s = t. 

— The labeling function Iq on W is defined in an obvious way: 

lo{{a,s)) = a. 

As can be easily verified, ^ is a pre-order. Note that {a, s) ~ (/?, t) holds if 
and only if s = t. In other words, states of We are equally preferred in this order 
if and only if they correspond to the same admissible state of E. 

It turns out that the above defined reflexive model inherits many of the prop- 
erties of the original epistemic state. It will be shown, in particular, that suitable 
definitions of credulous and contraction inference with respect to reflexive models 
will give us inference relations that will coincide with the corresponding infer- 
ence relations determined by the source epistemic states. In this way, we will 
immediately obtain the completeness of reflexive possible worlds models with 
respect to these kinds of inference relations. 



5.2 Possible Worlds Semantics for Credulous Inference 

The following definition provides a characterization of credulous inference with 
respect to reflexive possible worlds models. 
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Definition 7. A credulously entails B in a reflexive possible worlds model W 
if either A does not hold in any state of W, or there exists a preferred state s 
supporting A such that all equally preferred states supporting A support also B. 
The set of conditionals that are credulously valid in W will be denoted by 

The above definition says, in effect, that A credulously entails i? in a reflexive 
possible worlds model if either A is false in all states of the model, or there exists 
a preferred state s satisfying A such that the classical implication A-^B holds 
in all admissible states that are equally preferred relative to s. 

To begin with, a straightforward check verifies that the above credulous in- 
ference satisfies all the axioms of a credulous inference relation. 

Lemma 1. For any reflexive possible worlds model W, is a credulous infer- 
ence relation. 

Now we are going to show that the above possible worlds models are adequate 
for representing credulous inference. To this end we can use the following result: 

Theorem 4. //E is an epistemic state, then the corresponding reflexive possible 
worlds model We determines the same credulous inference relation as E. 

Since any credulous inference relation is determined by some epistemic state, 
the above theorem immediately gives a completeness of credulous inference with 
respect to reflexive possible worlds models: 

Representation Theorem 1 An inference relation is credulous if and only if 
it is generated by some reflexive possible worlds model. 



Semantics for permissive inference. Using the duality of credulous and per- 
missive inference, we obtain the following characterization of permissive inference 
in terms of reflexive possible worlds models: 

Definition 8. A permissively entails B in a reflexive possible worlds model W 
if, for any preferred state supporting A there exists an equally preferred state 
that supports both A and B. 

Notice that if the classes of equally preferred states are singletons, then the 
above definition will coincide with the definition of preferential entailment in 
KLM models. In the general case, however, this definition gives a semantic char- 
acterization of permissive inference. Thus, the following result is actually an 
immediate consequence of the above representation theorem for credulous infer- 
ence: 

Representation Theorem 2 An inference relation is permissive if and only 
if it is generated by some reflexive possible worlds model. 
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5.3 Possible Worlds Semantics for Contraction Inference 

Finally, we will give a representation of contraction inference in terms of reflexive 
models. 

It turns out that contraction inference with respect to reflexive possible 
worlds models is definable as follows: 

Definition 9. B is a contraction consequence of A m W if either A holds in 
all states of W, or there exists a preferred state s supporting ~<A such that all 
equally preferred states support B. 

It can be easily verified that the above definition determines a simple con- 
traction inference relation. Moreover, reflexive possible worlds models are also 
adequate for contraction inference. As before, this can be shown by proving that 
the reflexive possible worlds model We corresponding to a given epistemic state 
E generates precisely the same contraction inference relation. 

Theorem 5. //E is an epistemic state, then its corresponding reflexive possible 
worlds model We determines the same simple contraction inference relation as 

E. 



Since any contraction inference relation is determined by some epistemic 
state, also in this case the above theorem gives a completeness of contraction 
inference with respect to reflexive possible worlds models. 

Representation Theorem 3 ~| is o contraction inference relation iff it is gen- 
erated by some reflexive possible worlds model. 



6 Conclusions 

The results described in this report show that many alternative notions of non- 
monotonic inference can also be given a possible worlds semantics. Though we 
are not inclined to assign too much importance to such interpretations (as com- 
pared with epistemic states) the possibility of such a possible worlds semantics 
opens some new opportunities and perspectives in studying these kinds of non- 
monotonic inference. Thus, possible worlds semantics are viewed by many as 
a canonical way of supplying semantics to logical notions. In this respect, the 
above results hopefully make the above non-standard notions of nonmonotonic 
inference more friendly and intelligible for a broad logical community. But most 
important, they suggest that well-developed methods of modal logic and possible 
worlds semantics could be used for studying these notions. In particular, there 
is a relatively straightforward way of constructing a (bi-)modal logic that would 
provide a syntactic description for reflexive possible worlds models. Then the 
above kinds of nonmonotonic inference can be described by certain formulas of 
such a modal logic. This is one of the topics for a further study. 
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Abstract. Temporal reasoning with nonlinear models of time have been 
used in many areas of artificial intelligence. In this paper we focus on the 
model of branching time which has been proven successful for problems 
such as planning. We investigate the computational complexity of the 
point algebra for branching time extended with disjunctions and show 
that there are exactly five maximal tractable sets of relations. We also 
give an improved algorithm for deciding satisfiability of the point algebra 
with a time complexity comparable to that of path consistency checking 
algorithms. 



1 Introduction 

Many artificial intelligence systems include a component of temporal reasoning 
and many formalisms exist for describing and solving such problems. Two well 
known temporal languages are Allen’s interval algebra and the point algebra 
for totally ordered time HS|. In these algebras the basic entities are time inter- 
vals and time points, respectively. The satisfiability problem can be decided in 
polynomial time for the point algebra while it is NP-complete for Allen’s interval 
algebra. Due to the computational hardness many attempts have been made to 
find tractable fragments of Aliens’s interval algebra. 

However, Allen’s interval algebra and other temporal algebras with a linear 
model of time is not sufficient in many applications. Instead, other models of 
time such as branching time have been suggested and studied in some detail. 
The model of branching time has proven especially successful for such appli- 
cations. McDermott H21 has demonstrated the inadequacy of using only linear 
models of time in planning applications. Several logics based on branching time 
such as CTL and CTL* have been investigated earlier, see eg. the tutorial by 
Emerson and Srinivasan [0|. Furthermore, the point algebra for branching time 
has previously been examined by Diintsch et al. 0 from an algebraic point of 
view. 

The point algebra for branching time has also been investigated from a com- 
putational perspective and fc-consistency (for arbitrary k) has been shown in- 
sufficient for determining satisfiability for branching time by Hirsch HH. Hirsch 

* This research has been supported by the ECSEL graduate student program. 
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has also presented an algorithm running in O(n^) time for deciding satisfiability 
for this algebra. This can be compared to many other relational algebras where 
path consistency usually decides consistency for the tractable sets of relations. 
For instance, the point algebra for totally ordered time and all tractable 
sets of relations for the point algebra for partially ordered time 0 as well as 
all known tractable sets of relations for RCC-5 0 and RCC-8 M have this 
property. 

In this paper we examine the point algebra for branching time from a compu- 
tational perspective. We extend it with disjunctions for increased expressibility 
and give a complete classification of tractability. We do this for several reasons. 
First, simple constraint languages extended with disjunctions have been shown 
to have interesting properties and several examples of this have been given by 
Cohen et al. |0|. Secondly, disjunctions can compactly describe complex relations. 
Consider for example the ORD-Horn algebra m which contains 868 different 
relations. Defining it with the aid of disjunctions is very easy: ORD-Horn con- 
tains exactly the Allen relations which can be expressed by disjunctions of the 
form Xi < y\y X 2 yf 2/2 V . . . V ^ Vn- Thus, tractability of ORD-Horn fol- 
lows trivially from tractability of the point algebra for linear time extended with 
disjunctions. This approach has also proven successful for the point algebra for 
partially ordered time; a total classification of tractability has been given by 
Broxvall and Jonsson 

The main result of this paper is a total classification of tractability in the 
point algebra for branching time extended with disjunctions. Our results show 
that there exist exactly five maximal tractable sets of relations. Using these 
sets of relations we are not only able to solve problems containing these rela- 
tions efficiently but can also backtrack upon these sets of relations for problem 
instances containing arbitrary relations and disjunctions. It should also be possi- 
ble to form tractable sets of relations for the interval algebra for branching time. 
Additionally we give an improved algorithm for the full point algebra running in 
0(nM(n)) time where M(n) is the time complexity of multiplying two n x n ma- 
trices. Coppersmith and Winograd 0 have shown that there exists an algorithm 
for matrix multiplication running in time. By using their matrix mul- 
tiplication algorithm, our algorithm gets a time complexity of which 

should be compared to that of path consistency checking algorithms, O(n^). It 
is thus a significant improvement over Hirsch’s algorithm. 

Many relational algebras have been extended with disjunctions and the time 
complexity have been investigated. It is commonly found that extending a 
tractable set of relations containing disequality with disjunctions of disequality is 
again a new tractable set of relations. For instance, the point algebra for partially 
ordered and totally ordered time 0, RCC-5 and RCC-8 |S| have this property. 
Hence, one would expect tractability of the full point algebra for branching time 
extended with disjunctions of disequality. By showing that the opposite holds, we 
have one more example of the peculiar computational properties of this algebra. 

The paper is structured as follows: Section 2 contains basic definitions and 
some auxiliary results used in the tractability and maximality proofs. This is 
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followed in Section 3 by the tractability results we need. We continue in the 
following section by giving NP-hardness proofs to yield a base of NP-complete 
problems which in turn is used in Section 5 to show our maximality result. 
Finally, the last section contains a brief summary of our results. 

2 Preliminaries 

The basic computational problem for the point algebra for branching time is that 
of deciding satisfiability. We begin by recalling the general constraint satisfaction 
problem CSPSAT. The satisfiability problem CSPSAT(5) for sets S of relations 
over a domain I) is defined as follows: 

Instance: A set V of variables over a domain I) and a finite set 0 of constraints 
(i?, Xi, , Xn), where i? G 5 is a relation of arity n and all Xi G V. 
Question: Is there a total function f : V ^ T> such that for each constraint 
(i?, Xi,. . . , Xn) G 0 the following holds: (f{xi), . . . , f{xn)) G R. 

The size of a problem instance can either be regarded as the total number of 
variables and constraints or (as is common in eg. path consistency) simply the 
total number of variables. We choose the later approach and define the size of 
a problem instance as the total number of variables since this gives a stronger 
result than the alternative definition. 

For the point algebra for branching time, we are interested in CSPSAT prob- 
lem instances over a specific domain and allow only certain relations. We write 
SATbr(T) to denote CSPSAT problem instances over this domain and with 
relations only from F. We define the domain as the points in the forest 
containing all finite trees infinitely many times. We choose this definition rather 
than restricting all points to have a common ancestor since it simplifies the proofs 
and otherwise change nothing. The basic relations over points in this domain are 
denoted by <, >, = and ||. Given arbitrary points x, y in we say that: 

1. X < y iS X precedes y in 

2. X > y iS y precedes x in 

3. X = y iS x,y are the same point. 

4. x\\y iff x^y belong to different branches or trees. 

We will also refer to the point algebra for totally ordered time in some proofs. 
We write SATto(T) to denote the CSPSAT problem instances of this domain 
where only relations from F are allowed. The set of basic relations for SAT to is 
the relations <,>,=. 

Example 1. In the subset of (depicted in Figure P) some of the relations 
holding between points are: a < c, b\\c, b\\e, a < d and c\\f. 

We take unions of the basic relations to form new binary relations. Sometimes 
we will use a short hand notation for such relations. We write eg. <, || and T 
instead of < U =, || U = and < U > U || U =, respectively. We also use 
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Fig. 1. A small subset of 



the operators composition (71 072), intersection (71 IT 72) and converse (7”^) of 
binary relations in the usual way. The composition table for the point algebra 
for branching time can be found in Table E 

Given a problem instance U of variables V and constraints C we say that a 
total function f : V ^ ^hr interpretation of 77 . Furthermore, if / satisfies 
the constraints C then / is said to be a model of 77 . To simply the proofs we 
need one further concept. A point n € related to some point in the image 
of / by < or > but not itself in the image is said to be a redundant point. If 
there exists no redundant points for /, ie. if / satisfies the following: 

f(V) = {a e T>^j.\3x €V : a < f{x) V f{x) < a} 

we say that / is a non-redundant model of 77 . Note that if 77 has a model, 
then 77 also has a non-redundant model since it is always possible to remove an 
arbitrary point from a tree and preserve the relations between remaining points. 

Example 2. Let 77 be a problem instance over five variables Xq, Xi, X2, X3, X4 and 
the constraints: xq < xs,X2 yf X3,X4\\x3,X2 < X4,X4 < X2, xi\\x4, xq <> xi and 
xq <> X 4 . Furthermore let, /i,/2 and /a be the following interpretations: 

fi{xo) = a, f2{xo) = f, fsixo) = a 

fi(xi) = 6 , /2(a:i) = g, f3(xi) = g 

fl{X2) = dj2{X2) = h,f3{x2) = d 

fiixs) = e, 72(3:3) = i, f3{x3) = e 

fl{x4) = d,f2{x4) = h,f3{x4) = d 

where a, ... ,i are the point in given in Figure 0 We have that fi and /2 
are models of 77 but /a is not since the Xq <> xi constraint is unsatisfied by 
/a- Furthermore /2 is a non-redundant model while /i is redundant since c is 
connected to the nodes of /i but not itself part of the image of fi . 

Next, we define the disjunction operator which enables us to form more 
general constraints from the basic relations. 

Definition 1 . Let 7 ?i, 7?2 be relations of arity i,j and define the disjunction 
7 ?i V T?2 of arity i + j as follows: 

i^i V i?2 = {(^ 1 , ■ • ■ . . .,Xi)GRi\/ 

(^z+l 7 • ■ • ; ^ R-2 } 
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Table 1. The composition table for the point algebra for branching time 
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To give a concrete example of how the disjunction operator is used consider 
the following: 

Example 3. Let D = {0,1} and the relations And = {(1,1)} and Xor = 
{(0, 1), (1, 0)} be given. The disjunction of And and Xor is: 

r (0,0,0,1),(0,1,0,1), (1,0,0, 1), (1,1,0,!),} 

And V Xor = <^ (0, 0, 1, 0), (0, 1, 1, 0), (1, 0, 1, 0), (1, 1, 1, 0), J 

[ ( 1 , 1 , 0 , 0 ), ( 1 , 1 , 0 , 1 ), ( 1 , 1 , 1 , 0 ), ( 1 , 1 , 1 , 1 ) J 

We see that the constraint x And y\/x Xor z encoded by (And V Xor, x, y, x, z) 
is satisfied when a;, y and z have been instantiated to, for instance 1, 0, 0 respec- 
tively. 

We continue by defining the disjunction over sets of relations. The disjunction 
of two sets of relations is the set containing the disjunction of every pair of 
relations in the original set as well as the original relations themselves. It is 
sensible to include the original relations since one wants to have the choice of 
using the disjunctions or not. 

Definition 2. Let Ti ,/2 and A be sets of relations. We define disjunetions of 
two sets of relations Av /2 and disjunetions over a single set of relations Z\*, 
A* as follows: 



T 1 VT 2 = El U E2 U {i?i V i?2 I Ri G Ti, i?2 G T 2 } 

A^ = A Z\*+i = Z\*5Z\ 

OQ 

A* =\JA^ 

i=l 

Finally, we introduce the 1-independence property as defined by Co- 
hen et al. 1^. For simplicity we write independence rather than 1-independence. 
This concept will be used extensively for showing tractability results. 

Definition 3. For any sets of relations E and A, we say that A is indepen- 
dent with respect to E if the following holds: For any set of constraints C in 
CSPSAT(T U A), C has a solution whenever every C C C , which contains at 
most one constraint whose constraint relation belongs to A has a solution. 
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Table 2. Tractable classes 
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Example 4 . We see that {=} is not independent of {=, since the constraints 
C : X = y,y = z,x ^ z is not satisfiable although the constraints C' : x = y,x ^ 
z and C" : y = z,x ^ z are both satisfiable. Note that {=} is independent of itself 
since all problem instances containing only equality constraints are satisfiable. 

The following theorem proven by Cohen et al. jS| provides a useful mechanism 
for determining satisfiability of disjunctions of relations. 

Theorem 1. CSPSAT(TvZ\*) is tractable for any sets of relations E and A, 
such that CSPSAT(TUZ\) is tractable and A is independent with respect to E. 

3 Tractability Results 

In this section we will show that the sets of relations defined below (where 
Ea, . • . , Ae are defined in Table EJ are tractable. The same sets of relations will 
also be proven to be the unique maximal sets of tractable relations for SATbr 
in Section 5. 

Ta = Ea Tb = Eb^A*e Tc = A*c 
Td = EdvA*e Te = EevAe 

We begin by showing tractability of 1~a and by proposing an algorithm run- 
ning in 0{nM{n)) time, where M(n) denotes the time complexity of multiplying 
two nxn matrices. The algorithm can be found in Figure 0 Using the 
algorithm for matrix multiplication proposed by Coppersmith and Winograd [7] 
we see that our algorithm is a significant improvement over the 0{n^) algorithm 
given by Hirsch El. 

We use the notion of components in the algorithm in the following sense: 
X, y are in the same component iff there exists a path between x and y. Note 
that the concept of components are only defined for undirected graphs. In order 
to simplify the algorithm we assume that the set of constraints for problem 
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1 Algorithm BRANCH(iT) 

2 Input: Problem instance U = {V, C) of Fa 

3 Let G be the undirected graph {V, 0) 

4 for each constraint {R, x,y) € C such that || 2 ^ do 

5 add the edge {x, y} to G 

6 Partition G into components Vi, ... ,Vk 

7 Partition G into Gi, . . . ,Gk such that: 

8 Gi = {{R,x,y) G C\x,y G Vi} 

9 for each component Vi do 



10 


Let Gi be the directed graph (Vi, 0) 




11 


for each c = {R, x, y) G Gi such that R do 


12 


add the (directed) edge {x, y) to Gi 




13 


Let M be the adjacency matrix of the transitive 


14 


and reflexive closure of Gi 




15 


N G- new empty matrix indexed by Vi 




16 


for each {R, x, y) G Gi such that =2 R 


do 


17 


N[x,y] G- 1 




18 


M-N 




19 


root G- T 




20 


for each x G Vi do 




21 


if 'iy G Vi : P[x, {/] = 0 V M[x, y] = 0 


then 


22 


root G- X 




23 


if root 2 -L then 




24 


Vl ^ Vi - {c|M[root,c] = 1} 




25 


G'i^{{Rx,y)GCi\x,yGVl) 




26 


if Branch((V/, C()) rejects then reject 


27 


else reject 




28 


accept 





Fig. 2. Algorithm for determining satisfiability for Fa 



instances are closed under converse. We also use expressions such as <% R for 
which relations should be considered as sets of tuples, eg. we have <C<. 

The algorithm works by first partitioning the problem instance into sets of 
variables which can be mapped to disjoint trees, ie. all constraints between the 
partitions include the relation ||. This is done by partitioning the undirected 
graph G which contains an edge {a;, y} iff there exists a constraint {R, x, y) 
disallowing || into its components. Next, the algorithm tries to identify a variable 
root which can be mapped to the root node for each partition. Note that there 
exists exactly one such node for each satisfiable partition since each partition 
maps to a distinct tree. 

Let x\ be a candidate to be mapped to the root node in a satisfiable problem 
instance. If there exists a chain of constraints X\ R\ X2 ■ ■ ■ x„-i Rn Xn such 
that <2 Ri we must also map X2,...,x„ to the root node. Existence of the 
constraints xi Ri X2 ■ ■ ■ Xn-i Rn Xn for each pair of variables xi,Xn is checked 
by looking at the adjacency matrix M of the transitive closure for the graph 
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Gi constructed by the algorithm for every partition Vi. li M[x,y\ = 1 and x is 
mapped to the root node then so must y. 

Thus, for X to be mapped to the root node there must not exist a pair of 
variables which both are identified with x and for which we have a constraint dis- 
allowing the equality relation, ie. there must not exist nodes y, z and a constraint 
c = {R,y,z) such that = % R, M[x,y] = 1 and M[x,z] = 1. Conveniently, exis- 
tence of z and c such that = % R and M[x, z] = 1 can be computed by matrix 
multiplication. This is done in step El where P\x, y] = 1 iff there exists such 
a variable z and constraint c. After identifying a variable root which can be 
mapped to the root node, every variable which must be identified with root is 
removed from the partition and the algorithm is recursively called for this new 
problem instance. 

Before proving correctness of the algorithm we recall the definition of matrix 
multiplication. 



Lemma 1. If Algorithm Branch rejects a problem instance then it is not sat- 
isfiable. 

Proof: Assume to the contrary that there exists a satisfiable problem instance 
n which is rejected by the algorithm. Then there exists a component j of a 
subset of the original problem instance for which the algorithm rejects on line 
El Note that rejection on line ESI only occurs if the algorithm rejected on EZl 
after some recursions. Let 77' denote the problem instance {Vi,Ci) where Vi,Ci 
is the variables and constraints making the algorithm reject on line El Trivially, 
the algorithm rejects also 77'. Note that 77' is a subinstance of 77 and, hence 
satisfiable. 

Since 77' is satisfiable there exist a non-redundant model / of 77'. Assume 
that there exists more than one minimal point in the image of 77' under / and 
let f{x) and /(y) denote two such points. Note that minimal points in 
are the roots of distinct trees. Since the graph G constructed by the algorithm 
for 77' contains only one component then there exists a path of constraints 
X Ri xi ■■■ Xn Rn+i y such that II 2 Ri, ■ ■ ■ ^Rn+i- Since f{x) and /(y) are 
the root of different trees we obviously have some Xi,Xi+i whose images lies in 
different trees thus violating the constraint Xi Ri+\ Contradiction, and we 

have only one minimal point which must be the root of all other points in the 
image in 77'. 

Let X denote a variable in 77' such that f{x) is the minimal point. Since the 
algorithm rejects we know that there exists some y & Vi such that P[x, y] > 1 
and M\x,y] = 1. From the definition of matrix multiplication (OJ follows the 
equations 0 and m which obviously are equivalent. 



C = A- C[i,j] = 77^=1 A[*, k] ■ B[k,j] 



( 1 ) 




(2) 



zGVi 



3z G Vi : M[x, z] = 7V[z, y] = 1 



( 3 ) 
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Hence, we have z G 14, c = (R,z,y) G Ci such that M[x,z] = 1 and = % R. 
Since M[x,y] = 1, we have a chain of constraints x R\ yi R 2 ■■■ Rn+i y 
such that <2 Ri- Note that f{x) < f{yi) and hence, f{x) = f{yi) and 
fiVi) < fiVi) which gives f{x) = f{yi) = f(y 2 ) and /(?/ 2 ) < f{.yi) etc. This 
leads to f{x) = f{y^) = f{y). Analogously, f{x) = f{zi) = f{z). Contradiction, 
since f{z) = f{x) = f{y) violates the constraint c. Thus, II' and II are not 
satisfiable. □ 

Having proven that the algorithm rejects only unsatisfiable instances we will 
now demonstrate that the algorithm correctly identifies the satisfiable problem 
instances. 

Lemma 2. Algorithm Branch only accepts satisfiable problem instances. 

Proof: We prove the result by induction over the number of variables in the given 
problem instance. Obviously, all problem instances accepted by the algorithm 
containing zero variables are satisfiable. Assume that acceptance of the algorithm 
implies satisfiability for all problem instances of size n or less. Let 7T be a problem 
instance of size n + 1 which is accepted by the algorithm. We construct an 
interpretation / of the problem instance as follows: 

1. For each component i, let /' denote a model of {Vf , Ci). Note that \Vf \ < \V\ 
since at least one variable is removed on line Existence of /' follows from 
the induction hypothesis. 

2. For each component i, introduce a fresh root node t to each /'. We define 
an interpretation fi of {Vi,Ci) as follows: 

f ^ iff M[r-oot,a:] = 1 

\ fi {x) otherwise 

3. Define / as the union of the (disjoint) interpretations fi. 

We continue by demonstrating that each constraint C = (R,x,y) is satisfied by 
/. Assume x € Vi and y &Vj. We have two cases. 

Case 1: If i = j there are four further cases to analyze. If M[root,x] = 
M[root,y] = 0 then C is included in {Vi,Ci) and hence, satisfied by /',/i and 
thus /. When M[root,x] = M[root,y] = 1 we have P[root,y] = 0 which implies 
N[x, y] = 0. Hence, we have = C i? and thus, C is satisfied by fi and /. Finally, 
when M[root,x\ = 1 and M[root^y\ = 0 we have < Q R since there otherwise 
would exist a path from root to y in the graph Gi constructed by the algorithm. 
Hence, fi and / satisfies C. The case M[root,x] = 0 and M[root,y] = 1 can be 
shown analogously. 

Case 2: i f j. Since the algorithm has partitioned x and y into different 
components we know that || C i? and c is satisfied by /. □ 

By demonstrating that the algorithm runs in polynomial time we can now 
conclude that the satisfiability problem for 7 a is tractable. This is done in the 
following theorem. 
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Theorem 2. SATbr(TA) tractable. 

Proof: We have previously shown that algorithm Branch correctly decided sat- 
isfiability for SATbr(TA) problem instances. It is now only a matter of demon- 
strating that algorithm Branch runs in polynomial time. We begin by noting 
that the initial graph partitioning can be performed in 0{nf) steps by applying 
a standard graph partitioning algorithm, see eg. Baase |2|. Also, the transitive 
closure of Gi on line can be computed in the same time as the boolean ma- 
trix multiplication on line ^10. Hence, steps EEEI can all be performed within 
0{M{ni)) time. The following polynomial is an estimate of the total running 
time of algorithm Branch: 

f{n) < citi^ + AITi {c 2 M{rii) + f{rii - 1)) 

for some constants Ci,C 2 and where Ui denotes the size of component Vi. Note 
that we have rii + . . . + rik = n. Furthermore, for all c > 0 we have that n\ + 

I-Rfc < (riH \-nkY. Hence, S^^.^C 2 M{ni) < C 2 M{SYini) = C 2 M{n). This 

gives us: f{n) < (ci -I- C 2 )M{n) + S’f^^f{ni — 1) which can be shown to evaluate 
to a polynomial with positive coefficients by an induction over n. Hence, 

/(n) < (ci -I- C 2 )M{n) + f{n - 1) < (ci -I- C 2 )nM{n) 

□ 

We will now proceed to the four other tractable sets of relations. We begin 
by noting that Tb and Tc is tractable. 

Theorem 3. The satisfiability problem for Tb o-nd Tc is tractable. 

Proof: The same sets of relations have previously been investigated and proven 
tractable for the point algebra for partially ordered time We note that 
every problem instance of SATbr(TB) and SATbr(Tc) is satisfiable iff it has a 
totally ordered model, ie. a model / such that for all x,y holds f{x) <=> f{y). 
Hence, we can solve SATbr(7s) and SATbr(Tc) problem instances as problem 
instances of the point algebra for partially ordered time, which can be done in 
polynomial time. □ 

In our next theorem we demonstrate the tractability of To- 

Theorem 4. The satisfiability problem for To is tractable. 

Proof: The following two steps is all that is needed for an algorithm determining 
satisfiability for problem instances II of SATbr(Fb). 

1. For each constraint c = {=,x,y), remove c and replace all occurrences of y 
in n with X. 

2. If there exists a constraint {R, x, x) such that = ff- R then reject, else accept. 

If the algorithm above rejects an instance, it is clearly not satisfiable. Otherwise, 
it is satisfiable since all remaining variables can be mapped to root nodes in dis- 
joint trees. By an analysis of the algorithm it is obvious that Aq is independent 
oi Pd and hence, Po^Af, is tractable. □ 
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Theorem 5. SATbr(TE) is tractable. 

Proof: First, let P'^ = {<,^, ||}. Broxvall and Jonsson 0 have constructed an 
algorithm running in polynomial time for deciding satisfiability of SATbr(F^) 
which is similar to the Branch algorithm presented in this paper. By a straight- 
forward analysis of this algorithm, it can be proven that ^ is independent of P'^ 
and hence, is tractable. 

Next, a polynomial reduction from Pe instances to P'^ instances has been 
given by Broxvall and Jonsson |3|. By analyzing these proofs, it is evident that 
we can reduce to in polynomial time. □ 

4 Intractability Results 

This section contains the NP-completeness results which are needed for the main 
result. Note that for all sets of relations considered in this paper, the satisfiability 
problem is in NP. For the NP-hardness proofs we need the definition of the NP- 
complete problem 3-Colourability ED]: 

Instance: Undirected graph G = {V, E) 

Question: Is G 3-colourable, i.e., does there exist a function / : U — >■ {1,2,3} 
such that V(m,u) G E : f{u) ^ f{v) 

There are large similarities between the four NP-completeness proofs pro- 
vided in this section. Although the first one can be thought of as a general 
template for the construction of the other NP-completeness proofs we provide 
the other proofs since they give a good insight into the branching time model. 

Lemma 3. SATbr({|l V ||,<=>}) is NP-complete. 

Proof: We exhibit a reduction from 3-Colourability. Arbitrarily choose an 
undirected graph G = {V,E) such that V = |ui,---,u„}. We construct an 

instance 77 of SATbr({|| V ||, <=>}) which is satisfiable iff G is colourable with 

three colours. For each vertex Vi add the following constraints: 

ai <=> bi <=> Ci <=> a' (4) 

a' <=> 6' <=> c' <=> ai (5) 

a,|la' V bi\% , a,||a' V c,||c- , b,\% V c,||c- 

It can easily be seen that a problem instance given by the constraints (141,611 and 
n = 1 is satisfiable and remains so when we add one or two, but not three, of 
the constraints /(a)||/(a'), /(6)||/(6'), /(c)||/(c'). Since the problem instance 77 
constructed so far is satisfiable we have that for every model / of 77, exactly 
one of fiai)\\f{a'f),f{bi)\\f{b'^) or /(ci)||/(c') does not hold. Our intention with 
the construction is that for each variable Vi, it holds that /(a^) <=> f{a[) iff 
Vi can be coloured with the first colour and so on for the other “colours” bi and 
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Ci . Note that a vertex may have a choice of several colours in which case any of 
the colours can be chosen. For each edge (vi,Vj) in G add the constraints: 

Oilla- V aj\\a'j , 6,||6' V b^\\b'j , c,||c- V Cj||c' 

which ensures that Vi and Vj are not assigned the same colour. The resulting 
set of constraints can be computed in polynomial time, it is an instance of 
SATbr({|| V !!,<=>}) and it is satisfiable iff G is 3-colourable. □ 

Lemma 4. SATbr({yf V ||, <>}) is NP-complete. 

Proof sketch: By adding the constraints ai||a', 6i||6', Ci||c- to those given by 00 
in the proof of Lemma 0 we can substitute all occurrences of || by y^. □ 

Lemma 5. Let G {<>,<=>} then SATbr({i?i V i? 2 , ||}) is NP- 

complete. 

Proof sketch: The proof is very similar to the proof of Lemma0 For each vertex 
the set of constraints should contain: 

a\\b' , b\\c , c\\a 

a Ri b V b R 2 c b Ri c V c R 2 a c R\ a V a R 2 b 

Which enforces that at least one of the a, a', b, b' or c, d pairs will be related by 
the relation || which also determines the colour of the vertex. The constraints 
corresponding to the edges of G is of the same form as in lemma 0 but using the 
i?i and i ?2 constraints. □ 

Lemma 6. SATbr({|| V ||, <,<=>}) is NP-complete. 

Proof: Again, we make a reduction from the NP-complete problem 3- 

COLOURABILITY. Arbitrarily choose an undirected graph G = {V, E) such that 
V = {ui, • • • , Vn}. We construct an instance 7T of SATbr({|| V ||, <, <=>}) which 
is satisfiable iff G is colourable with three colours. For each vertex Vi, add the 
following constraints: 

ai <=> o', bi <=> 6', Ci <=> c'i , a'i < bi, b'i < Ci, c'i < Oi 

a^\\a', V bi\\b[ , a,\\a'^ V Cij|c- , h,\% V Cij|c- 

The problem instance constructed so far is satisfiable and for each model / 
exactly two of the following relations holds: /(oi) = /(a'),/(6i) = f{b'f) or 
f{ci) = f{c'f). Note that the variable pair not related by equality will be related 
by <> because of the three first constraints. Again, our intention is to assign 
colours to the vertices by the models constructed for II. We say that a variable 
Vi can be coloured by the first colour when /(a^) yf f{a'f) and so forth for bi and 
Ci. For each edge (vi,Vj) in G add the constraints: 

ai\\a'i V a^a' , bf\\b[ V 6J6' , c4c- V c^c' 
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which ensures that Vi and Vj are not assigned the same colour. The resulting 
set of constraints can be computed in polynomial time, it is an instance of 
SATbr({|| V II, <, <=>}) and it is satisfiable iff G is 3-colourable. □ 



Theorem 6. The satisfiability problem is NP-eomplete for the following sets of 
relations: 

1 . {<}?{<} 

{=}v{=} U {i?} where R. 

3. {i?i}v{i?2} U {< II) <} where i?i, i?2 are <> or <=>. 

4- ill V II, <},{|| Vjl, <, <=>} and V yf,||, <=>} 

Proof: For the two first points we note that SATto({<}v{<}) 

SATto({=}v{=} U {R}) have been proven NP-complete previously by Broxvall 
and Jonsson The reduction to the corresponding branching time satisfiability 
problem is trivial. The remaining sets of relations have been proven NP-complete 
in this section. □ 



5 Maximality 

We will now demonstrate that the five sets of relations proven tractable in Section 
3 are the only maximal tractable sets. In order to do so we need a concept of 
maximality for relational algebras with disjunctions and we use the definition in 
Broxvall et al. |5I • Let T be a set of disjunctive relations constructed from a set 
B of binary relations by applying the v operator. We say that T is a maximal 
traetable set iff P is tractable and for every set X P of relations which can be 
constructed by the relations in B and v, PUX is intractable. We need a number 
of equivalence results. 

Lemma 7. The following problems are equivalent up to polynomial-time reduc- 
tions. 

1. SATbr(T) and SATbr(T0 where T' is the closure of P with respect to con- 
verse, composition and intersection. 

2. SATbr({< VA}UT) ond SATbr({< Vi?,AVi?}UT). 

3. SATbr({= VA,i?'}ur) ond SATbr({= Vi?,i?',AVi?}UT) where =% R' . 

I SATbr({|| Vi?,i?'}ur) andSATbr({|| where R' is <> 

or <=>. 

5. SATbr({|[Vi?,< ||,i?'}ur) and SATbr({|[ V i?, < ||, i?', i? V i?}) where R' 
is <> or <=>. 

6. SATbr({yf V R, ||, A} U P) and SATbr({yf V yf V i?, \\,R} U P) whenever 
R is the relation <> or <=>; 

7. SATbr({7i V 72,71 V 73} ur), SATbr({7i V72, 7i V73, 7i V(72073)} U T) and 
SATbr({7iV72,7iV73,7iV(72n73)}UT). 
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Proof: Correctness of the first point is trivial. The two following points have 
been proven previously by Broxvall and Jonsson ^ for the point algebra for 
partially ordered time. By examining these proofs it is obvious that they hold 
also for branching time since the reductions from NP-hard problems constructed 
in the proofs are satisfiable for partially ordered time iff they are satisfiable for 
branching time. For the fourth point consider the following set of relations: 

a R' b , b R' c , c R' d , x Ry\/ a||c , z RwV b\\d 

and note that at most one of a||c and b\\d can hold. Hence, x R y V z R w 
must hold without further restricting the choices of x,y,z,w. We use this in 
order to make a polynomial reduction from SATbr({|l V R, R', RV R} U P) to 
SATbr({|| V R, R'} U r). Let 77 be an arbitrary problem instance of the first set 
of relations and replace each 7? V 7? constraint of 77 with the constraints above 
where a, 5, c, d are fresh variables. Clearly the resulting set of constraints is an 
instance of SATbr({|| V R,R'} U P) that is satisfiable iff the original instance is 
satisfiable. Correctness of the next two points follows by similar reasoning. The 
final point follows from the following two equivalences. 

— X y\/ z (72 n 73) w is equivalent to a: 71 y V z 72 w and x y V z w 

— X y\/ z (72073) w is equivalent to a; 71 y V z 72 t and a; 71 yV t w where 

t is a fresh variable. 



□ 

We define the closure operation C{X) as the maximal set of relations con- 
structible by successive applications of the previous rules. Note that C{X) is 
tractable iff X is tractable and C{X) is NP-complete iff X is NP-complete. 

For the main result of this section, we use a construction which simplifies 
the proof by allowing us to consider only a finite number of disjunctions; this is 
shown in the next lemma proven by Broxvall and Jonsson 

Definition 4. Let T = PvA*. We define T as {T — P) VJ {P — A)v{P — A) 
where T denotes the set of all binary relations eomposed by the union of basie 
relations from this domain. 



Lemma 8. 7/ T = PvA* , A C P and T' 2 then there exists C € T sueh 
that C gT' ■ 

We can now prove the main result of this paper by using a computer assisted 
case analysis working in a similar way as the maximality proof of Broxvall and 
Jonsson 

Theorem 7. Ta,Tb,Tc,Td and Te are the only maximal tractable disjunctive 
subclasses o/SATbr- 

Proof: Suppose to the contrary that there exists another maximal tractable 
algebra T. From the previous lemma it follows that there exists 'Ja, ' ' ‘ ,1e in 7” 
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such that & T Ar ‘ ' ^lE & Te- Note that there exists only a finite number of 
possible values for 'ja, ■ ■ ■ ,Je- 

To prove the result, a machine-assistecG case analysis of the following form 
was performed: each admissible choice of 7 a, ■■■,7b was generated and T = 
C(7a, ■ ■ ■ , 7e) was computed^ Each such set T was examined and it was found 
that at least one of the NP-complete sets of Theorem 0 was a subset of T- Thus, 
SATbr(T) is NP-complete and the theorem follows^ □ 

6 Summary 

We have identified five tractable sets of relations for the point algebra for branch- 
ing time extended with disjunctions and by using a computer assisted case anal- 
ysis shown these to be the only maximal tractable sets^ We have also given a 
revised algorithm for determining satisfiability for the point algebra with a time 
complexity comparable to that of path consistency checking algorithms ■ Previ- 
ously only algorithms running in O(n^) time have been known for this algebra^ 
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Abstract. In a previous paper we proposed an approach to exploit lit- 
eral equivalences in connection tableau based calculi. There we showed 
that making equivalences explicit offers new possibilities for search space 
reduction by applying literal demodulation for simplification and by 
strengthening the well-known regularity refinement. In this paper we 
generalize this approach to handle conditional equivalences. The gener- 
alization is mainly motivated by the circumstance that non-conditional 
equivalences, if not present at the beginning of a deduction, are much 
harder to generate than conditional ones. 



1 Introduction 

An inference mechanism working on a specific problem representation usually 
has no access to the information that is implicitly included in the representa- 
tion of a formula. For instance, a formula in clausal form may implicitly contain 
information about equivalences that - if made explicit - can be used for simplifi- 
cation based on the fact that equivalence on literals is closely related to equality 
of terms. Given a literal-equivalence A = B, any occurrence of literal B can be 
replaced by literal A in case A < B holds for some Noetherian ordering. Such a 
use of equivalences provides some of the power of equality reasoning techniques 
for problems not noted in terms of equality, thus it partially solves a question 
stated by Wos: ’’What is the appropriate theory for demodulating across ar- 
gument and across literal boundaries ... to replace certain predicates by other 
predicates and certain collections of literals by other collections? ” H2|. 

In this vein, a calculus with logical equivalence was proposed in m- It com- 
bines resolution with the possibility to derive literal-equivalences and to use them 
as rewrite rules. It was shown that the reduction part of a corresponding calculus 
can be considerably improved by literal demodulation. In |^, a related approach 
was presented for the connection tableau calculus which is the basis for success- 
ful proof systems like Setheo PI and KoMeT Since the connection tableau 
calculus is a top-down backward-chaining calculus ~ unlike saturation calculi 
based on resolution the situation is more complex because a straightforward 
application of demodulation would result in an incomplete calculus. However, it 
was shown that a careful exploitation of equivalences is feasible and can result in 
enormous reductions of the search space Q These reductions are due to simplifi- 

^ For instance, the resulting calculus is able to decide clause sets of the form 
{{p{x),p{f{x))},{~^p{x),-'p{f'^{x))}} within seconds. Without using equivalences 
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cations achieved by demodulating the set of input clauses and by strengthening 
the regularity refinement. For generating equivalences dynamically in the course 
of a deduction, 0 also proposes a technique which is based on the analysis of 
non-regular tableaux (see Section 0 for more details). 

The approaches presented in m and |S] share one basic restriction: Only 
the simplest form of equivalences - namely literal equivalences - is considered. 
Clearly, such equivalences are useful since they can be directly used for simplifi- 
cation. They are however - if not present at the beginning of a deduction ~ quite 
difficult to generate. Therefore, we propose an approach which is based on condi- 
tional equivalences, i.e. formulas of the form Ci, . . . , Cn A = B, which are, as 
we shall show in Section El comparatively easy to derive. To exploit conditional 
equivalences, we present two techniques: The first - and conceptual simpler - 
one uses lemmata (which are used in many proof systems for avoiding derivation 
duplication) to generate non-conditional equivalences from conditional ones. The 
second one aims at using conditional equivalences in situations where not enough 
information is available to turn them into non-conditional ones. We show that a 
conditional equivalence (of the above form) can be employed in a local fashion, 
that is, roughly speaking, it can be used for simplification during subderivations 
of some goal -iG in case it is guaranteed that in order to find an overall refuta- 
tion, subrefutations for ~<C\, . . . , ~<Cn have to be generated using a proof context 
that equals (or is smaller than) the one that is available for refuting -iG. 

The rest of the paper is organized as follows. Section 13 provides some ba- 
sic terminology and summarizes the approach pursued in |S| . In Section 0 we 
demonstrate the difficulties in deriving non-conditional equivalences and present 
an approach to generate conditional equivalences. Sections 0 and 0 are devoted 
to the aforementioned techniques for exploiting conditional equivalences. In Sec- 
tion 0 we conclude. 

2 Handling Non-conditional Equivalences 

In this section we first provide some additional terminology concerning equiva- 
lences and show how literal-equivalences can be used for demodulation. After- 
wards we introduce the basic concepts of the calculus presented in 0 . 

In what follows, we assume the reader to be familiar with the fundamental 
concepts of first-order clause logic. As usual, the variables occurring in clauses 
are considered implicitly as being universally quantified, a clause is considered 
logically as a disjunction of literals, and a clause set is taken as a conjunction 
of clauses. The letters u,v,w,x,y,z are used to denote variables, letters a,b,c 
are used to denote constant symbols. Let L'^ denote the negation of a literal L, 
i.e. if L is an atom then = -iL, otherwise if L = -lA then L‘^ = A. L and 
L‘^ are called complementary literals. In addition to the standard definitions of 
atomic formulas and literals, we shall deal with atomic formulas that consist of an 
equivalence (we simply use the word equivalence instead of literal-equivalence) . 

this is a hard problem for top-down backward-chaining proof systems. SETHEO 

(version 3.0) EH is not able to find a proof in less than one honr for n = 20. 



124 S. Bmning 



Definition 1 (E-literal, Rule). An E-literal is a pair (L,K) where L and K 
are literals. If the pair is ordered, then the E-literal {L, K) is also called a rule, 
and written in the form L i— >■ K . Otherwise, it is written in the form L = K . We 
shall, however, still write {L,K) to include both possibilities L = K and L K. 

For an E-literal E = (L,K), we define Cl{E) = {{L, K‘^}, {L^^, K}} and for 
a set £ of E-literals we set Cl{£) = Cl{E). We sometimes use the term 

equivalence instead of E-literal if this cannot lead to confusion. 

Given a set of E-literals £, L =£ L' denotes that L is equivalent to L' wrt £, i.e. 
there is a sequence {Li,Ki), . . . , (L„, K^) of elements of £ and a substitution a 
such that L = Licr, Kia = Li+icr for all 1 < i < n, and = L' . In this case, 
we also say that L is £- equivalent to L' .li £ = 0, then L =£ L' in case L = L' . 

Using a reduction ordering, it is possible to direct E-literals to rules. A rule 
R = L ^ K can be used to reduce literals in clauses as well as in E-literals: If 
for some substitution a, La = L' , then R reduces L' to Ka. Since L = K is 
logically equivalent to = K‘^, a rule used to reduce some literal L' is assumed 
to be in the form L ^ K where L' and L have the same sign. An E-literal of the 
form {L,L) is called trivial. An E-literal E is subsumed by another E-literal E' 
if E'a = E for some substitution a. Subsumed E-literals and trivial E-literals 
can always be removed. 

In order to check a set of E-literals for unsatisfiability, so-called L-para- 
modulation steps are used. L-paramodulation steps allow to deduce new E-literals 
and correspond to a kind of superposition in the terminology of rewriting. 

Definition 2 (L-Paramodulation Step). Let E = {L,K) and E' = {L',K') 
be E-literals. If there is a substitution a such that La = L'a, the E-literal 
{Ka,K'a) is an L-paramodulant of E and E' . If there is a substitution r such 
that L‘^t = Kt, then the empty clause is an L-paramodulant of E. 

In what follows, we introduce a connection tableau based calculus augmented 
by a refined handling of (non-conditional) equivalences. For brevity, we do not 
introduce the pure connection tableau calculus (e.g. see ini) in a first step. 
Instead, we directly present the enhanced calculus given in 0. Differences to 
the pure connection tableau calculus are explicitly pointed out. 

The calculus presented in does not only rely on connection tableaux as 
basic proof objects. Instead, a theorem to be proven is divided into a set of input 
clauses S and a set of E-literals £ where we assume that S is irreducible wrt £. 
The clauses in S are used for generating so-called connection E-tableaux using 
the derivation steps known from the pure connection tableau calculus, namely 
initialization, extension, and reduction steps. For the sake of completeness, an 
additional derivation step, called equivalence step, is required, which allows to 
modify E-tableaux using the elements of £. Further, so-called L-Demodulation 
steps are used to reduce - during a deduction - the sets S and £ as well as the 
actual E-tableau by dynamically generated equivalences (as we will see later, 
equivalences can be generated by inspecting non-regular E-tableaux). 

Definition 3 (Connection E- Tableau). A clausal tableau is a downward- 
oriented tree in which all nodes except the root node are labeled with literals. 
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For every non-leaf node N in a clausal tableau, the sequence N\, . . . , Nm of its 
immediate successor nodes is called the successor sequence of N; if the nodes 
are labeled with literals Li, . . . , L^, respectively, then the clause {Li , . . . , is 
termed the tableau clause below N. The tableau clause below the root node is 
called top-clause. 

Let S be a clause set and £ be a set of E-literals. A clausal tableau T is said 
to be an E-tableau of S wrt £ if for every tableau clause {Li, . . . , Ln} in T, 
there is a substitution a and a clause {Ki, . . . , iC„} € S such that Li =s KiU 
(1 < i < n). An E-tableau is called connected or connection E-tableau if each 
inner node labeled with a literal L has a leaf node among its immediate successors 
which is labeled with literal L‘^. 

A branch of a connected E-tableau T is a sequence Ni, . . . , Nn of nodes in T 
such that Ni is the root node, Ni is the immediate predecessor of Ni^i {1 < i < 
n), and is a leaf. For i > 1, Ni,. . . ,N^ is called a subbranch starting with 
Ni. A (sub)branch 5 is closed if it contains two nodes labeled with complementary 
literals. Otherwise, b is open and the literal attached to its leaf node is called an 
open goal. An E-tableau is closed if each of its branches is closed. 

The distinguishing feature of connection E-tableaux in contrast to ordinary con- 
nection tableaux (see m is that a tableau clause needs not to be an instance 
of an input clause. Instead, it merely has to be equivalent to such an instance. 

Definition 4 (Initialization Step). At the beginning, select a clause from S 
and take it as top-clause of the E-tableau. 

Definition 5 (Extension Step). Select a leaf node N of an open branch labeled 
with literal L. Let {Li,...,L„} be a new variant of a clause in S such that 
there exists an mgu a with L'^a = Lia for some 1 < i < n. Then, attach n 
new (immediate) successor nodes N\, . . . , Nn to N , label them with L\, . . . , Ln, 
respectively, and apply a to all tableau literals. We call Ni an extension node, 
and the elements o/{iVi, . . . , — non-extension nodes. A literal attached 

to an extension node is called extension literal, otherwise non-extension literal. 
Eurthermore, Lj is called the original literal of Nj for all 1 < j < n. 

Definition 6 (Reduction Step). Select a leaf node of an open branch labeled 
with literal L. Lf there is a node on the same branch labeled with literal L' such 
that there exists an mgu a with L‘^<j = L'a, then apply a to all tableau literals. 

A calculus which generates E-tableaux using initialization, extension, and reduc- 
tion steps is sound and complete for first-order clause sets. In case E-literals are 
stored separately (in set £), a further derivation step, called equivalence step, is 
required which allows to replace open goals by f-equi valent literals. 

Definition 7 (Equivalence Step). Select a leaf node N of an open branch 
labeled with literal L. Lf there is a set {{Li, Ki), . . . , (L„, A'„)} of variants of 
elements of £ and an mgu a such that La = L\a and Kia = Ti+icr for all 
1 < i < n, then apply a to all tableau literals and label N with K^a. 
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Fig. 1. E-tableaux for Example^ Leaf nodes of closed branches are underlined. 



Example 1. For illustration, consider the following clause set sB 

(1) g(/"-(a)) ^ (2) q{v) ^p{v) (3) p{u) q{f{u)) 

(4:)p{y) ^p{f{y)) (5) p{f{z)) ^ p{z) (6) ^p(a) 

Clauses (4) and (5) encode the E-literal E = {p{f{y)),p{y)) which can be di- 
rected to a rule R = p{f{y)) >->■ p{y)- Rewriting S with R, clauses (4) and (5) 
become tautological and are removed. Now consider the E-tableaux in Figure Q 
The first one is built by an initialization step with clause (6), the second one re- 
sults from an equivalence step with n — 1 instances of E. An extension step with 
clause (3) generates the third E-tableau, the last (closed) E-tableau is generated 
by a further extension step with clause (1). 



Remark 1. The need for equivalence steps is due to the connectedness criterion 
given in Definition 0 One might object that this makes a refined handling of 
equivalences useless. This is fortunately not the case since the search for ’mini- 
mal’ refutations by demodulating the set of input clauses may reduce the search 
space considerably. For more details and illustration, see |0|. 

In view of an implementation one should take into consideration that the 
application of one equivalence step encodes several implicit extension steps using 
clauses from Cl{£). The number of these implicit steps has to be taken into 
account if some kind of iterative-deepening search strategy is performed. 



Definition 8 (E-Derivation, E- Refutation) . A sequence D of initialization, 
extension, reduction, and equivalence steps generating a connection E-tableau T 
from a clause set S and a set of E-literals £ is called an E-derivation for S and 
£. If T is closed, D is caZfed E-refutation. 

Let L be an open goal attached to a node N in a connection E-tableau T. 
A subderivation D for N (or L) is an E-derivation where the first element of 
D selects N and each further element selects a descendant of N . D is called a 
subrefutation if after applying D to T, each branch containing N is closed. 

In many cases we write clauses in a PROLOG like manner. 
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The number of connection tableau derivations to be taken into account can be 
restricted considerably by using the regularity restriction (e.g. see El) which 
allows to ignore E-tableaux containing two identical literals on one branch. Inter- 
estingly, the use of equivalences allows to strengthen the regularity restriction. 
Instead of requiring two identical literals on one branch to be applicable, the 
literals only have to be logically equivalent. 

Definition 9 (E- Regular E- Tab lean). Let £ be a set of E-literals. An E- 
tableau T is E-regular wrt £ (or f-regular ) if no two different nodes on a branch 
are labeled with literals that are £ -equivalent. 

Remark 2 . In case f = 0, E-regularity equals “classical” regularity. Since it is 
in general undecidable whether two literals are equivalent, one should - in order 
to keep the E-regularity check simple - only test whether a branch contains two 
literals that can be reduced to the same literal. For more details, see 0. 

Example 2 . We continue Example Q After an initialization step with clause (6) 
two extension steps with clauses (3) and (2), respectively, are applied. The re- 
sulting E-tableau has one open branch which contains the literals ~'p{a) and 
-•p{f{a)). Thus, the E-tableau is not E-regular wrt {{p{f{y)),p{y))}, and there- 
fore the derivation is pruned. In the same way, any E-derivation applying ex- 
tension steps using clauses (2) and (3) to generate the term /”(a) is pruned. 
Therefore, the only possibility to derive a closed E-tableau is to apply an equiv- 
alence step (as in Example 0 . This reduces a search space of exponential size 
(in n) to one of linear size. Note, that in case only the “classical” regularity 
restriction is employed, the size of the pruned search space remains exponential. 

The following theorem is the main result of jOl . 

Theorem 1. Given a clause set S and a set of E-literals £, SU Cl{£) is unsat- 
isfiable iff there exists an E-refutation for S and £ generating a closed £ -regular 
E-tableau, or the empty clause is derivable from £ by L-paramodulation steps. 

So far, we have not discussed the possibility to exploit equivalences which are 
generated during a deduction (see Section 0. If a set of new E- literals is derived, 
the current set of E-literals as well as the set of input clauses can be reduced. 
As a consequence, the current E-tableau T may not be any longer an E-tableau 
of the reduced clause set. Then, some of the derivation steps generating T have 
to be withdrawn. This is accomplished by a so-called L- demodulation step. 

Definition 10 (L-Demodulation Step). Let S be a set of clauses, £ be a 
set of E-literals, and T be a connection E-tableau of S wrt £ generated by an 
E-derivation d\, . . . ,dn. Let £' be a set of E-literals that is logically implied by 
5U Cl{£). An L-demodulation step consists of three successive substeps: 

(i) Reduce £\J £' to an irreducible set £" (that is no rule in £'' can be used 
to reduce another E-literal in £" ). (ii) Reduce S wrt £'' to an irreducible set S" . 
(Hi) Take derivation steps back (starting with dn) until the resulting E-tableau 
is an E-tableau of S" wrt £" . 
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Fig. 2. E-Tableaux for Examples ^ ^ and 0 Leaf nodes of closed branches are 
underlined. 



After the application of an L-demodulation step, the deduction process continues 
with the sets S" and E” . Note that an L-demodulation step is a simplification 
step. It has not to be taken back during the entire deduction. In [3| it is shown 
that L-demodulation steps preserve correctness and completeness. 

3 Generation of Conditional Eqnivalences 

Besides providing a calculus for exploiting (non-conditional) equivalences, jO) 
also contains an approach for generating new equivalences during a deduction. 
This is useful in those cases where equivalences are only included implicitly in 
the (clausal) representation of a problem. The basic idea of [HI is to generate 
new equivalences from E-tableaux which are not E-regular. 

Example 3. Consider a clause set S containing (among others) the clauses 

(1) p{x) ^ q{f{x)) (2) q{y) ^ r{y) (3) r(/(z)) ^ p{z) 

and an E-derivation for S generating the open goal ->p{a). Applying three ex- 
tension steps using clauses (1), (2), and (3), respectively, results in a non-regular 
E-tableau T (the first one depicted in Figure Ej). This E-tableau makes the cy- 
cle of implications included in S explicit what allows to derive the E-literals 
{p{x),q{f{x))), {q{f{x)),r{f{x))), and {p{x),r{f{x))) (note that the substitu- 
tion of the variables in T by constant a is due to the initial open goal ~'p{a)). 

However, this approach has one - quite severe - restriction: Let Ni,...,Nn 
be an open subbranch where Ni and Nn are labeled with equivalent literals. 
Then, in order to generate new E-literals from the corresponding E-tableau, 
each subbranch starting in A^i except the one ending in iV„ must be closedH 

® An in its essence similar restriction holds for the resolution based calculus presented 
in m- There, in order to generate a literal equivalence two binary clauses have to 
be resolved. This requires the previous derivation of ’’suitable” binary clauses. 
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Example 4- We continue Example 0 and replace clause (2) with clause 
(2') q{y) -ir- r{y),s{y). As before, we can generate a non-regular E-tableau by 
applying three extension steps using clauses (1), (2'), and (3), respectively. The 
resulting E-tableau T' is the second one depicted in Figure 0 Since T' contains 
the open goal -is(/(a)), it is no longer possible to generate E-literals. 



Remark 3. One might argue that the situation given in the previous example 
changes if another goal selection strategy is used throughout the deduction. Then 
-•s{f{a)) might be selected before -ir(/(a)) which would allow to derive E-literals 
after generating a subrefutation for -is(/(a)). It is very likely, however, that such 
a different selection strategy applies ’wrong’ selections in other situations. 

The approach pursued in this paper is to overcome this problem by deriving 
conditional equivalences, that is, formulas of the form Ci A . . . A — >■ {A, B) 

where, roughly speaking, Ci, . . . , C„ correspond to (the negations) of the open 
goals that prevent the generation of non-conditional equivalences. Reconsidering 
Example 0 where we have one such open goal, namely ->s(/(a)), it should be 
possible to generate s{f{x)) {p{x) , q{f (x))) , s{f{x)) -)> {q{f{x)),r{f{x))), 

and s{f{x)) — >■ (p{x),r{f{x))) (note again that the substitution of the variables 
in T' by constant a is due to the initial open goal ->p(a)). 

In what follows, we use the following terminology: A formula E of the form 
Cl A . . . A C„ — >■ {A, B) is called conditional E-literal if n > 0. In accordance with 
Definition 0 we use the terms conditional E-literal and conditional equivalence 
interchangeably if no confusion is possible. If n = 0, we often call E a non- 
conditional equivalence (instead of equivalence or E-literal) . Ci , . . . , C„ are called 
the condition literals of E. E is called trivial ii A = B. E is subsumed by an 
E- literal E' if there is a substitution a such that E'a = {A, B). 

To formalize the generation of new conditional E-literals from E-tableaux 
that are not E-regular, we use so-called subtableaux. Basically, a subtableau of 
an E-tableau T is a subtree of T. We use subtableaux to isolate those parts of 
a derivation which can be used for the derivation of (conditional) equivalences. 
Importantly, the overall substitution required to build a subtableau of T is usu- 
ally more general than the overall substitution required for the generation of T. 
This gives us the possibility to derive more general (conditional) equivalences 
than by inspecting the literals of T directly. 

Definition 11 (Subtableau). Let T be an E-tableau generated by an E- 
derivation D and let Ni, . . . ,Nn be the non-extension nodes of a successor se- 
quence in T. The subtableau of T rooted with N\,. . . , Nn is generated as follows: 

Eirst, an initialization step with top-clause {Ti, . . . , T„} is performed, where 
Li is the original literal (see Definition EP of Ni (1 < j < n). The resulting E- 
tableau contains n leaf nodes Mi , . . . , Mn labeled with Li, . . . , Ln, respectively. 
Afterwards, the extension and equivalence steps in D that where applied to Ni 
(and its successors) are applied to Mi (with possibly different mgus), for all 
1 < i < n. A reduction step is carried over only in case no ancestor of Ni was 
used to apply the reduction step. 
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Example 5. We continue Example El Let N denote the node in T' labeled with 
literal ->q{f{a)). The subtableau T" of T' rooted with N is depicted in Figure 0 
on the right. Note, that the variables in T" are not bound to constant a. 

Theorem 2. Let T\ he an E-tableau of a clause set S wrt a set of E-literals S , 
and Ni, . . . , Nn be some open subbranch of Ti. Let Ui, . . . , Um be the successor 
sequence of Ni with Ui being an extension node. Further, letT 2 be the subtableau 
of T\ rooted with U 2 , ■ ■ ■ , Um and a be the overall substitution of the derivation 
generating T 2 . // the literals attached to Ni and Nn are E -equivalent, then 

S U Cl{£) h Cfr A ... A Cfr — >■ L 2 T =s L 3 T A ... A Ln-iT =s LnX, 

where (i) for 2 < i < n, Li is the literal attached to the node in T 2 which 
corresponds to node Ni in Ti, (ii) Ci,...,Ci are the open goals in T 2 except 
Ln, and (Hi) t is a substitution such that Lfar =£ LnX where L\ denotes the 
original literal of U\ . 

Note that the label L oiU\ in Ti is the complement of the literal attached to N\ 
in Ti and that L is an instance of Li since no equivalence step can be applied 
to extension nodes. 

In case the literals attached to and iV„ in Ti - say L[ and - are equal, 
we obtain r as the mgu of Lfa and L„. Otherwise, the equivalence of L'l and 
can be shown via a sequence of E-literals (Ki,K [), . . . , {Kr, K() (see Section0). 
Then, r is the mgu such that Lfar = Kipr and K(pT = where p is the 
mgu which unifies K[ and for 1 < i < r. 

Remark 4 . There are two quite obvious generalizations of Theorem 0 The first 
one is to demand that iVi and some Nj with j < n (instead of Ni and Nn) 
are labeled with equivalent literals and to generate (conditional) equivalences 
from the literals attached to Ni, . . . , Nj. The second one is to take an additional 
substitution 7 into account and to generate (conditional) equivalences from E- 
tableaux where the labels of and Nj are f-equivalent after the application 
of 7 . For the sake of simplicity, however, we will stick to Theorem 0 the first 
generalization would, for instance, require a generalization of Definition 1 1 II 

Example 6 . We continue Example 0 In terms of Theorem 0 we have: T\ and 
T 2 are the second and third E-tableau depicted in Figure 0 respectively, tr = 
{y\f{x),z\x}, and r = 0. Following Theorem 0we get 

S U Cl{£) h s{f{x)) -)> p{x) = q{f{x)) A q{f{x)) = r{f{x)) A p{x) = r{f{x)) 

what results in the conditional E-literals s{f{x)) — >■ (p{x),q{f{x))), 
{q{f{x)),r{f{x))), and s{f{x)) {p{x),rlf{x))) . 

The exhaustive use of Theorem 0 might lead to the generation of a huge number 
of conditional E-literals@ Therefore, we recommend the usage of heuristics to 

Similar problems occur in case lemmata are generated in a non-restricted manner, 
see |1I13| . 
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reduce this number. One such heuristic is to avoid conditional E-literals with 
more than one or two condition literals. In case non-ground derivations (that 
generate tableaux containing many non-ground open goals) are considered, an- 
other heuristic is to ignore ground conditional E-literals since their applicability 
is limited in such cases. 



4 Lemma Handling and Conditional Eqnivalences 



Lemma handling (e.g. see 1111 31101 ') is a basic technique to avoid derivation 
duplication in top-down backward-chaining calculi, like model elimination or 
connection tableau calculi. The simplest - albeit most useful - form of lemmata 
are so-called unit lemmata that consist of only one literal. Roughly speaking, 
they can be derived in case an E-tableau contains a closed subtableau. Given a 
unit lemma L, which is generated during an E-derivation for a clause set S and 
a set of E-literals £, it is guaranteed that S U Cl{£) h L holds H 

In this section we show that besides using unit lemmata for avoiding deriva- 
tion duplication, they are also useful for generating non-conditional equivalences 
from conditional ones. Since most proof systems based on the connection tableau 
calculus (e.g. Setheo 1 1 4| 1 3j or KoMeTg]) already employ lemmata such a com- 
bination is quite attractive. 

In what follows, the set of conditional E-literals generated throughout a de- 
duction is denoted by C£, the set of derived unit lemmata is denoted by C. 
We assume that C£ as well as £ are reduced wrt £. Basically, a non-conditional 
equivalence (i.e. an E-literal) can be obtained from a conditional E-literal in case 
each of the condition literals (or a more general literal) has been proved as unit 
lemma or is already contained as unit clause in the set of input clauses. 



Theorem 3. Let S he a clause set, £ be a set of E-literals, C£ be a set of 
conditional E-literals, and C he a set of unit lemmata such that the elements of 
C£ and £ are logical implications of S U Cl{£). 

Let Cl A . . . A — >■ {A, B) be an element ofC£ and Li, ... ,Ln be a sequence 

of literals such that for 1 < i < n, Li G C or {Li} G S holds. Lf there is a 
substitution a such that Ci<j = Lia for 1 < i < n, then S U Cl{£) h Aa = Ba. 

An integration of this approach for handling conditional equivalences into the 
calculus presented in Section 0 requires the following extensions: 

— An initialization step has to initialize the sets C£ and £ to the empty set. 

— Unit lemmata and conditional E-literals, that are generated throughout the 
deduction have to be reduced wrt £ and added to £ and C£, respectively. 

— Definition iini(L -demodulation step) has to be extended as follows: (iv) The 
sets C£ and £ have to be reduced wrt £" to sets C£" and £", respectively, 
and (v) elements of C£” have to be deleted if they are trivial or subsumed 
by elements of £” . 

® There is no difference for the generation of lemmata between "ordinary” connection 
tableau derivations and E-derivations as defined in this paper. 
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Note that in case an E-literal {Aa^ Ba) is generated according to TheoremOl the 
corresponding conditional E- literal is deleted by a following L-demodulation step 
from C£ only in case {Aa, Ba) and {A, B) are equal up to variable renaming. 

Example 7. We continue Example 0 As we have seen, we can add the con- 
ditional E-literals s{f{x)) -)■ {p{x),q{f{x))), s{f{x)) -)> {q{f{x)),r{f{x))), 
and s{f{x)) — >■ {p{x),r{f{x))) to C£. In case is it possible to derive unit 
lemma s(/(a)) afterwards, Theorem 01 allows us to generate the set of E-literals 
£[ = {(p(a),g(/(a))),(g(/(a)),r(/(a))),(p(a),r(/(a)))}. More useful, i.e. 

more general E-literals can be generated in case a more general 
lemma, for instance s{y), can be deduced. Then, we get £2 = 

{{Pix),q{f{x))),{q{f{x)),r{f{x))),{p{x),r{f{x)))}. Reducing C£ wrt (what 
happens by the application of an L-demodulation step) , the previously generated 
conditional E-literals become trivial and can be removed. 

Remark 5. One might object that in case unit lemmata are employed, there is 
no need for handling conditional equivalences at all. For illustration reconsider 
ExampleEl If it is possible to deduce a unit lemma of the form s{f {...)) or s{y), 
one might argue that it should be possible to derive (non-conditional) E-literals 
directly. However, this is only true if such a unit lemma is generated before the 
second E-tableau (see Figure EJ. Otherwise, in case no conditional E- literals are 
generated, the possibility to derive non-conditional ones might be lost since the 
same situation might not occur again (after the generation of the unit lemma) . 
This holds even if backtracking is taken into account since the applicability 
of lemmata might prevent subderivations that are generated if lemmata are not 
present. For instance, in case lemma p{a) is available to close a branch with open 
goal -'p(a), there is no need to consider alternative subderivations for -ip(a). 

Another argument in favor of handling conditional equivalences is the possi- 
bility to derive different non-conditional E-literals from one conditional E-literal 
(depending on the applied unit lemmata as illustrated in ExamplejJ)- In 0, gen- 
erated E- literals only depend on the respective tableaux that violate E-regularity. 
Reconsidering Example Q this may lead to a situation where only the set of E- 
literals £[ can be derived instead of the more useful set £2 ■ 

5 Using Conditional Equivalences for Local Simplification 

In this section we present an approach that aims at exploiting conditional equiv- 
alences in situations where not enough information - like unit lemmata - is avail- 
able to generate non-conditional equivalences. Roughly speaking this approach 
is based on the idea that given some conditional E- literal Ci A . . . A(7„ — >■ (L,K), 
E-literal {L, K) can be used if it is guaranteed that in order to find an overall 
refutation, subrefutations for Cf, . . . , must exist, too. For preserving sound- 
ness, it is essential to apply (L, K) in a local fashion: If (L, K) is applied during 
a subderivation of some goal G, it must be guaranteed that G has at least the 
same preeonditions (i.e. ancestor literals for the application of reduction steps) 
that are available to find subrefutations for Cf,..., 
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Definition 12 (Precondition). Let L be a literal attaehed to an node N in a 
eonneetion E-tableau. The set of literals attaehed to the aneestors of N is ealled 
the precondition of L and denoted by Pre^. 

The precise conditions that have to be met in order to use a conditional E-literal 
like a non-conditional one are given in the following definition: 

Definition 13 (Justification for Conditional E-Literals). Let G be an open 
goal in a connection E-tableau T and let E = Ci A . . . A Cn -A (L, K) be a condi- 
tional E-literal. If for some substitution a, T contains besides G non-extension 
literals Gfa, . . . ,Gf^a, such that (i) for 1 < i < n, Pre^d^ C Prec, and (ii) 
var{Ci<j A ... A Gn<j) H var{{La, Ka)) = (10, then G has a justification for Ea. 

Condition (ii) of the above Definition assures, that substitutions which may be 
required to complete the subrefutations of Gfa, .. ., G^a do not affect the ap- 
plicability of {La, Ka) during subderivations of G (note that Gfa, . . . , G!^a may 
be inner literals, they do not have to be open goals). Without this condition it 
might be the case that {La, Ka) is used for simplification although the subrefu- 
tations for Gfa, . . . , Gtfa require an additional substitution r that only allows 
the usage of an instance of {La,Ka) (namely {Lar, Kar)). 

Theorem 4. Let E = C\ A ... A Gn ^ {L, K) be a conditional E-literal and 
G be an open goal which has a justification for E. Then, {L, K) can be used 
throughout subderivations of G (without sacrificing soundness or completeness). 

Remark 6. Obviously, the usefulness of the above theorem increases with the 
number of ground non-extension literals that are generated during a deduction. 
However, even if a derivation generates lots of non-ground open goals, it is of- 
ten the case that these non-ground goals become ground by the application of 
derivation steps. Further, many application domains consider restricted classes of 
clause sets that only allow derivations where all - or at least most - open goals 
(and therefore also all - or at least most - (inner) non-extension literals) are 
ground. For instance, this is the case for (some kind of) range-restricted clause 
sets that are used by connection-tableau based calculi for default reasoning j3 
0 or in the area of deductive databases 0- 

The application of Theorem E| requires some form of ’’local” L-demodulation 
steps: In subderivations of G it is possible to use {L,K) for simplification and 
equivalence steps. Outside subderivations for G this might be impossible since 
the respective open goals might not have a justification for E. Hence, the notion 
of E-tableaux (Definition El has to be generalized: An E-tableau cannot be de- 
fined any longer wrt to a fixed set of input clauses. Instead, it must be allowed 
to contain tableau clauses that are equivalent to instances of input clauses which 
are reduced wrt E-literals that can only be applied locally. Due to lack of space 
we have to skip the corresponding formal definitions. The following example 
illustrates our approach. 



By var{E) we denote the set of variables occurring in E. 
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Fig. 3. Tableaux for Examples El and 0 Leaf nodes of closed branches are underlined. 



Example 8. Let 5 be a set containing (among others) the following clauses: 

{1){A,B,C} {2){A,B,^C} (3){A,-B,C} 

(5) {-A B, C} (6) {-A, -C} (7) {-71, -B, C} 

Consider an E-tableau of S containing a tableau clause cl with open goals —A 
and —C as depicted in Figure 0on the left. Applying two extension steps with 
clauses (4) and (6), the second (non-regular) E-tableau depicted in Figure Olcan 
be generated which allows to derive the conditional E-literal E = C ^ {A, B). 

Since literal —A contained in tableau clause cl has the same precondition 
than literal —C (in cl), —A has a justification for E. Hence we can use E-literal 
E' = (A, B) during subderivations for —A. Assuming that A < B for some 
reduction ordering < holds, we direct E' to a rule i? H> A and simplify S. The 
resulting set contains instead of clauses (1) - (7) the following ones: 

(l')|A,C} (2'){A,-C} (7'){-A,C} 

Now it is possible to find a short subrefutation for —A consisting of two exten- 
sion steps with clauses (!’) and (2’) (see the third E-tableau in Figure |3). A 
subrefutation that makes no use of equivalences needs at least 5 extension steps. 

It should be noticed that the technique presented in Section 0] might not be 
applicable in the given situation, even if another goal selection strategy is used: 
Since a subrefutation for —C might depend on ancestor literals of —C, it might 
be impossible to find a subrefutation for —C that allows the derivation of unit 
lemma C . 

The next example shows that in order to use a conditional E-literal E during 
subderivations of some goal G, it is necessary that G has a justification for E. 

Example 9. We continue Example|H|and replace clause (5) by clause {—A, B, D}. 
As before, it is possible to generate the conditional E-literal E = G ^ (A, B) 
and to find a short subrefutation for —A. On the other hand, it is easy to verify 
that no classical subrefutation (i.e. a subrefutation that only uses extension and 
reduction steps) for the remaining open goal -<G can be found. Now, assume 
that (A, B) can be used safely during subderivations of —C (although ->G has 
no justification for E). Then, for the sake of soundness, it should not be possible 
to find a subrefutation for —C using clauses (!’), (2’), and (7’). This however is 



Exploiting Conditional Equivalences in Connection Calculi 



135 



possible by applying two extension steps with clauses (7’) and (1’), respectively. 
Hence, the application of {A, B) during derivations of -<C is unsound. 

Similar problems occur if a combination of the techniques presented in this and 
the previous section is considered. If both are combined, one has to guarantee 
that in case a (conditional) E-literal E (or a lemma L) is generated from a 
(sub)derivation that makes use of some conditional E-literal E' (according to 
Theorem^), then E (or L) must not be used during subderivations of goals that 
have no justification for E\ 

Finally, we mention that conditional equivalences can also be exploited for a 
refined handling of E-regularity: Whenever some open goal G has a justification 
for a conditional E- literal Ci A . . . A — >■ {A, B), no subbranch starting in G 

is allowed to contain literals that are equivalent wrt to f U {(H, B)} where - as 
above - £ denotes the set of E-literals used throughout the deduction. 



6 Conclusion 

We proposed an approach for handling conditional equivalences in connection 
calculi. It extends previous work, presented in |0|, where it was shown that a 
combination of connection calculi with the ability to exploit literal equivalences 
for simplification has a great potential to reduce the search space. The presented 
extensions are, on the one hand, a method for deriving conditional equivalences 
and, on the other hand, two mechanisms for exploiting conditional equivalences. 
The first of these techniques allows to turn conditional equivalences into non- 
conditional ones by a combination with lemma handling. The second one shows 
how conditional equivalences can be used directly in a local fashion for search 
space reduction. We argued that the exploitation of conditional equivalences is 
more promising in comparison to the approach presented in [HI since conditional 
equivalences are easier to generate than non-conditional equivalences. The afore- 
mentioned second technique further shows that conditional equivalences can be 
useful in situations where it is not possible to derive non-conditional ones. 

We do not claim that the use of (conditional) equivalences makes it possi- 
ble to find shorter proofs for any class of clause sets. In those cases where no 
equivalences are included (explicitly or implicitly) in the problem representa- 
tion, the presented mechanisms will be of no help. Even worse, their application 
will reduce the inference rate of the proof system. In other cases, however, the 
exploitation of equivalences allows to find proofs which are otherwise unobtain- 
able in reasonable time (for instance see Example 0 or Footnote 0). Hence, we 
recommend to implement the proposed techniques as an optional feature which 
can be switched on or off. Many calculi refinements - like lemma handling - are 
usually implemented in this way (since no technique is useful in all cases) . 

The results of this paper should be equally applicable to other top-down 
backward-chaining connection calculi, as they are presented in P|. Even an ex- 
tension of the resolution based approach in uni seems to be quite straightforward, 
with one exception: There is no direct way to carry over the technique presented 
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in Section 0 since it makes use of the proof structure given by connection E- 
tableaux (see Definitions Id and C3). 

An interesting direction to generalize our results is to allow arbitrary equiv- 
alences. For instance, it would be quite attractive to handle equivalences of the 
form bachelor(x) = male{x) A -<married{x) for problems in the area of Knowl- 
edge Representation (e.g. see 0). Approaches that use arbitrary equivalences 
(in combination with calculi that differ significantly from the connection tableau 
calculus) are presented in and PS). In both cases, however, completeness 
of the resulting calculi and the possibility to derive new equivalences are not 
considered. In order to extend the approach pursued in this paper, it seems to 
be necessary to take formulas in non-normal form (instead of clausal form) into 
account (e.g. see Pj). Another direction for future research is the integration of 
equivalences in confluent connection calculi (e.g. see | 2 |). In such calculi it should 
be possible to achieve completeness without equivalence steps. This would clearly 
enhance the effectiveness of using equivalences for demodulation. 
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Abstract. We show that propositional logic and its extensions can sup- 
port answer-set programming in the same way stable logic programming 
and disjunctive logic programming do. To this end, we introduce a logic 
based on the logic of propositional schemata and on a version of the 
Closed World Assumption. We call it the extended logic of propositional 
schemata with CWA {PS^ , in symbols). An important feature of the logic 
PS'^ is that it supports explicit modeling of constraints on cardinalities 
of sets. In the paper, we characterize the class of problems that can 
be solved by finite PS'^ theories. We implement a programming system 
based on the logic PS'^ and design and implement a solver for processing 
theories in PS'^ . We present encouraging performance results for our ap- 
proach — we show it to be competitive with smodels, a state-of-the-art 
answer-set programming system based on stable logic programming. 



1 Introduction 

Logic is most commonly used in declarative programming and knowledge rep- 
resentation as follows. To solve a problem we represent its constraints and the 
relevant background knowledge as a theory in the language of some logic. We 
formulate the goal (the statement of the problem) as a formula of the logic. We 
then use proof techniques to decide whether this formula follows from the theory. 
A proof of the formula, variable substitutions or both determine a solution. 

Recently, an alternative way in which logic can be used in computational 
knowledge representation has emerged from studies of nonstandard variants of 
logic programming such as logic programming with negation and disjunctive 
logic programming | pd'l’991Nie9H| . This alternative approach is rooted in seman- 
tic notions and is based on methods to compute models. To represent a problem, 
we design a finite theory so that its models (and not proofs or variable substitu- 
tions) determine problem solutions (answers). To solve the problem, we compute 
models of the corresponding theorjO This model-based approach is now often 
referred to as answer-set programming (or ASP). 

^ We commonly restrict the language by disallowing function symbols to guarantee 
finiteness of models of finite theories. In the present paper, we also adopt this as- 
sumption. 
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Logic programming with stable model semantics IHEHH! (stable logic pro- 
gramming or SLP, in short) is an example of an ASP formalism (MIM!- In SLP, 
we represent problem constraints by a fixed program (independent of problem 
instances). We represent a specific instance of the problem (input data) by a 
collection of ground atoms. To solve the problem, we find stable models of the 
program formed jointly by the two components. To this end, we first ground it 
(compute its equivalent propositional representation) and, then, compute sta- 
ble models of this grounded propositional program. Thanks to the emergence 
of fast systems to compute stable models of propositional logic programs, such 
as smodels [MSOOj . SLP is quickly becoming a viable declarative programming 
environment for computational knowledge representation. Disjunctive logic pro- 
gramming with the semantics of answer sets pTT) is another logic programming 
formalism that fits well into the answer-set programming paradigm. An effec- 
tive solver for computing answer sets of disjunctive programs, dlv, is available 
jFJ.M+98j and its performance is comparable with that of smodels. 

Our goal in this paper is to propose answer-set programming formalisms 
based on propositional logic and its extensions. Our approach is motivated by 
recent improvements in the performance of satisfiability checkers. Researchers de- 
veloped several new and fast implementations of the basic Davis-Putnam method 
such as satz and r els at [RWl . A renewed interest in local-search tech- 

niques resulted in highly effective (albeit incomplete) satisfiability checkers such 
as WALKSAT |bK094| . capable of handling large CNF theories, consisting of 
millions of clauses. Improvements in the performance resulted in an expanding 
range of applications of satisfiability checkers, with planning being one of the 
most spectacular examples [K IVIS9ti|KS99) . 

The way in which propositional satisfiability solvers are used in planning 
[IKMSDfij clearly fits the ASP paradigm. Planning problems are encoded as propo- 
sitional theories so that models correspond to plans. In our paper, we extend 
ideas proposed in |KMS96] in the domain of planning and show that propo- 
sitional satisfiability can be used as the foundation of a general purpose ASP 
system. To this end, we propose a logic to serve as a modeling language. This 
logic is a modification of the logic of propositional schemata |K MS9fi| : we explic- 
itly separate theories into data and program, and use a version of Closed World 
Assumption (CWA) to define the semantics. This logic is nonmonotonic. We call 
it the logic of propositional schemata with CWA (or, PS™°'). 

The logic PS^'"°‘ offers only basic logical connectives to help model problem 
constraints. We extend logic to support direct representation of con- 

straints involving cardinalities. Examples of such constraints are: ”at least k 
elements from the list must be in the model” or ’’exactly k elements from the 
list must be in the model” . They appear commonly in statements of constraint 
satisfaction problems. We refer to this new logic as extended logic of propositional 
schemata with Closed World Assumption and denote it by PS'^ . 

In the paper we characterize the class of problems that can be solved by finite 
PS^ theories. In other words, we determine the expressive power of the logic 
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PS^ . Specifically, we show that it is equal to the expressive power of function- free 
logic programming with the stable-model semantics. 

For processing, theories in PS~^ could be compiled into propositional theories 
and “off-the-shelf” satisfiability checkers could be used for processing. However, 
propositional representations of constraints involving cardinalities are usually 
very large and the sizes of the compiled theories limit the effectiveness of sat- 
isfiability checkers, even the most advanced ones, as processing engines. Thus, 
we argue against the compilation of the cardinality constraints. Instead, we pro- 
pose an alternative approach. We design a “target” propositional logic for the 
logic PS'^ (propositional logic PS'^). In this logic, cardinality constraints have 
explicit representations and, therefore, do not need to be compiled any further. 
We develop a satisfiability checker for the propositional logic PS~^ and use it as 
the processing back-end for the logic PS~^ . Our solver is designed along the same 
lines as most satisfiability solvers implementing the Davis-Putnam algorithm but 
it takes a direct advantage of the cardinality constraints explicitly present in the 
language. 

Experimental results on the performance of the overall system are highly en- 
couraging. We obtain concise encodings of constraint problems and the perfor- 
mance of our solver is competitive with the performance of smodels and of state- 
of-the-art complete satisfiability checkers. Our work demonstrates that building 
propositional solvers capable of processing of high-level constraints is a promising 
research direction for the area of propositional satisfiability. 

Our paper is organized as follows. In the next section we introduce the logic 
p^cwa — ^ fragment of the logic PS^ without cardinality constraints. We de- 
termine the expressive power of the logic in Section 0 We discuss the full 

logic PS^ in Section^ In the subsequent section we discuss implementation de- 
tails and experimental results. The last section of the paper contains conclusions 
and comments on the future work. 

2 Basic Logic 

Our approach is based on the logic of propositional schemata. The syntax of this 
logic is that of first-order logic without function symbols. The semantics is that 
of Plerhrand interpretations and models, which we identify with subsets of the 
Herbrand base. In the paper we consider only those theories in which at least one 
constant symbol appears. Among all formulas in the language, of main interest 
to us are clauses, that is, expressions of the form 

aiA...Aa„^BiV...VH„, (1) 

where each Oi is an atom and each Bj is an atom or an expression of the form 
3F&(s), where b{s) is an atom and T is a tuple of (not necessarily all) variables 
appearing in b{s). Each of m and n (or both) may equal 0. If m = 0, we replace 
the conjunct in the antecedent of the clause with a special symbol T {truth). 
If n = 0, we replace the empty disjunct in the consequent of the clause with 
a special symbol F {eontradiction) . We assume that each clause is universally 
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quantified and drop the universal quantifiers from the notation. We further sim- 
plify the notation by replacing each expression 31"6(s) in the antecedent by b{s'), 
where in s' we write a special symbol for each variable from Y in s. 

Let T be a finite theory consisting of clauses. For a formula B = 3F&(s) 
appearing in the consequent of a clause in T, we define B" to be the disjunction 
i?® = V ... V where s' , 1 < i < k, range over all term tuples that 

can be obtained from s by replacing variables in Y with constants appearing 
in T. Since T is finite, the disjunction is well defined (it has only finitely many 
disjuncts). 

For a clause C G T of the form dU, we define a clause C® by 

C® = aiA...Aam=>BfV...VB^, (2) 

A ground instance of C is any formula obtained from (7® by replacing every 
variable in C® by a constant appearing in T (different occurrences of the same 
variable must be replaced by the same constant). We define the grounding of 
T, gr{T) as the collection of all ground instances of clauses in T, except for 
tautologies; they are not included in gr(T). We have the following well-known 
result. 

Proposition 1. Let T be a finite clausal theory. Then a set of ground atoms M 
is a Herbrand model ofT if and only if M is a (propositional) model of gr(T). 

The language may contain several predefined predicates and function symbols 
such as the equality operator and arithmetic comparators and operations. We 
assign to these symbols their standard interpretation. However, we emphasize 
that the domains are restricted only to those constants that appear in a theory. 

We evaluate all expressions involving predefined function symbols and all 
atoms involving predefined relation symbols in the grounding process. If any 
argument of a predefined relation is not of the appropriate type, we interpret 
the corresponding atom as false. If a function yields as a result a constant that 
does not appear in the theory or if one of its arguments is not of the required type, 
we also interpret the corresponding atom as false. We then eliminate tautologies 
and simplify the remaining clauses by removing true “predefined” atoms from 
the antecedents and false “predefined” atoms from the consequents. 

Let us consider an example. Let T be a theory consisting of the following two 
clauses: 

Ci= q{b,c)^p{a) 

C 2 = p{X) ^ {3Yq{X, Y))y{X = a). 

There are three constants, a, b and c, and two predicate symbols, p and q, in the 
language. Symbols X and Y denote variables. The clause C 2 can also be written 
(using the simplified notation) as 

C2= p(W) ^g(A,_)V(A = a). 

To compute gr{T) we need to compute all ground instances of C 2 {C\ is itself 
its only ground instance). First, we compute the formula C|: 

= p{X) ^ q{X, a) V q(X, b) V q{X, c) V (A = a). 
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To obtain all ground instances of C2 (or Cf), we replace X with a, b and c. 
The first substitution results in a tautology (due to occurrence of ‘a = a’ in 
the consequent of the clause). Two other substitutions yield the following two 
ground instances of C (we drop atoms = a’ and ‘c = a’ from the consequents; 
they are false by the standard interpretation of equality): 

p(b) q{b, a) V q(b, b) V q{b, c) 

p{c) q{c, a) V q(c, b) V q(c, c). 

These two clauses together with Ci form gr{T). The sets of ground atoms 
{p{a),q(b,c)} and {p{b),p(c),q{b,a),q{c,c)} are two examples of models of T 
(or gr{T)). 

In order for the logic of propositional schemata to be useful as a programming 
tool, we modify it to separate input data from the program encoding the problem 
to be solved. We distinguish in the set of predicates Pr of the language a subset, 
Pr' . We call its elements data predicates. We assume that predefined predicates 
are not data predicates. All predicates other than data predicates and predefined 
predicates are called program predicates. A theory of our logic is a pair {D, P), 
where Z? is a finite collection of ground atoms whose predicate symbols are data 
predicates {data), and P is a finite collection of clauses (a program). 

To define the semantics for the logic, we use grounding and a form of CWA. 
We say that a set of ground atoms (built of data and program predicates) is a 
model of a theory {D, P) if 

Ml: M is a model of gr{D U P) (or, equivalently, M is an Herbrand model of 
D\J P), and 

M 2 : for every ground atom p{t) such that p € Pr' {p is a data predicate), 
p{t) G M ii and only if p{t) G D. 

We call the logic described above the logic of propositional schemata with 
CWA and denote it by PS™''. Due to (M 2 ), not every model of gr{D,P) is a 
model of {D,P). Consequently, one can show that our logic is nonmonotonic. 
This difference between the logic of propositional schemata and the logic PS™" , 
while seemingly small, has significant consequences for the expressive power of 
the logic and its applicability as a programming tool. 

Before addressing these two issues, let us consider an example. Let A and B 
be two disjoint and finite sets. We define D = {pi{a): a G A} U {p2{b). b G B}. 
We define P to consist of two clauses: 

Exl: qfiX) ^ pi(A) Ex 2 : 52(A) ^ p2{X). 

The constants are elements of A U P; A is a variable. The predicates are pi, 
P2, 5i and 52- The first two are data predicates. 

By (M 2 ), each model of a ps™" theory {D,P) contains D. However, it 
does not contain any ground atom pi{b), where b G B, nor any ground atom 
P2{a), where a G A. Each ground instance of the clause (Exl) is of the form 
5i(c) Pi{c), where c is a constant (c G A U B). Since Pi{c) G M if and 
only if c G A, it follows that if 51(c) G M, then c G A. Similarly, we obtain 
that if 5i(c) G M, then c G A. Thus, M is a model of {D,P) if and only if 
M = Did {51(a): a G A'} U {52(6): 6 G B'}, for some A' C A and B' C B. 
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Let us choose an element from A, say ag, and an element from B, say bg. Let 
us then add to P the clause 
Ex3: _pi(oo) ^ Pi{bg) 

We denote the new program by P' . The theory {D, P) has no models even 

though gr{D, P) is propositionally consistent. The reason is that all propositional 
models satisfying gr{D,P) contain pi{bg). Thus, none of these models satisfies 
condition (M2). This example illustrates that our semantics is different from cir- 
cumscription as circumscription preserves consistency. Circumscription applied 
to Pi would result in models in which the extension of pi in D would be min- 
imally extended by one more constant bg. Our (strong) minimization principle 
does not allow for any additions to the extension of data predicates. Intuitively, 
it is exactly as it should be. Data predicates are meant to represent input 
data. The program should not be able to extend it. 

Logic is a tool to model problems. To illustrate this use of the logic, 

we show how to encode the vertex-cover problem for graphs. Let G = (V,E) be 
a graph. A set IT C 1/ is a vertex cover of G if for every edge {x, y} & E, x or 
y (or both) are in W . The vertex-cover problem is defined as follows: given a 
graph G = (V, E) and an integer k, decide whether G has a vertex cover with 
no more than k vertices. 

For the vertex-cover problem the input data is described by the following set 
of ground atoms: 

Eve = {vtx{v):v G V}\J {edge{v,w)\ {u, w} G E}U {size{k)}U {pos{i): 1, . . . ,n}. 
This set specifies the set of vertices and the set of edges of an input graph. It 
provides the limit on the size of a vertex cover sought. Lastly, it uses a predicate 
pos to specify a range of integers that will be used to label vertices. The problem 
itself is described by the program Pyp 

VCl: vpos{I, X) ^ vtx{X) 

VC2: vpos\l,X) => pos{I) 

VC3: vtx{X) ^ vpos{-, X) 

VC4: vpos{I,X) A vpos{J,X) ^ I = J 
VC5: vpos\l, X) A vpos\l, Y)^ X = Y 

VC6: edge{X,Y) A vpos{I,X) A vpos{J,Y) A size{K) (/ < K) V ( J < K) 

(VCl) and (VC2) ensure that vpos{i,x) is false if i is not an integer from 
the set {1, . . . , n} or if x is not a vertex. (VC3)-(VC5) together enforce that the 
atoms vpos(i,x) that are true in a model of the PS‘^'^°' theory (Dyc,Pvc) define 
a permutation of the vertices in V. Finally, (VC6) ensures that each edge has 
at least one vertex assigned by vpos to positions 1, . . . ,k (in other words, that 
vertices labeled 1, . . . , fc form a vertex cover). The correctness of this encoding 
is formally established in the following result. 



Proposition 2. Let G = (V,E) be an undirected graph and let k be a positive 
integer. A set of vertices {rui, . . . , Wk} QV is a vertex cover of G if and only if 
M = D.UC U {vpos{i, Wi): i = 1, . . . ,k} is a model of the theory {Dye, Pvc)- 
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For another example, we will consider the n-queens problem, that is, the 
problem of placing n queens on a n x n chess board so that no queen attacks 
another. 

In this case, the representation of input data describes the set of row and 
column indices: 

Dnq = {vos{i): 1, . . . ,n}. 

The problem itself is described by the program Pnq- The predicate q describes 
a distribution of queens on the board: q{x,y) is true precisely when there is a 
queen in the position (x,y). 

nQl: q{R,C) ^ pos{R) 
nQ2: q{R,C) ^ pos\c) 
nQ3: q\R,Cl) ^ q{R,C2) ^ Cl = C2 
nQ4: q{Rl,C) ^ q{R2,C) ^ Rl = R2 
nQ5: q{R,C),q{R + I,C + I) 
nQ6: q{R,C),q{R + I,C - I) ^ P 

The first two clauses ensure that if q{r, c) is true in a model of (Dnq, Pnq) then 
r and c are integers from the set {1, . . . , n}. The following two clauses enforce the 
constraint that no two queens are placed in the same row or the same column. 
Finally, the last two clauses guarantee that no two queens are placed on the 
same diagonal. As in the case of the vertex cover problem, also in this case we 
can formally show the correctness of this encoding. 

These examples demonstrate that PS™°' programs can serve as represen- 
tations of computational problems. Two key questions arise: (1) what is the 
expressive power of the logic PS^'“°‘, and (2) how to use the logic as a 

practical computational tool. We address both questions in the remainder of the 
paper. 

3 Expressive Power of 

A search problem, 77, is given by a set of finite instances, Djj, such that for each 
instance 7 G D[j , there is a finite set Sn (7) of all solutions to 77 for the instance 
7 The graph-coloring, vertex-cover and n-queens problems considered in 

the previous section are search problems. More generally, all constraint satis- 
faction problems including basic AI problems such as planning, scheduling and 
product configuration can be cast as search problems. 

We say that a PS™°“ program P solves a search problem 77 if there exist: 

1. A mapping d that can be computed in polynomial time and that encodes 
instances to 77 as sets of ground atoms built of data predicates 

2. A partial mapping sol, computable in polynomial time, that assigns to (some) 
sets of ground atoms solutions to 77 (elements of U/gD /7 ^n{I)) 

such that for every instance 7 G D[j, s G Sn{I) if and only if there exists a 
model M of the PS'^™°‘ theory ((7(7), P) such that M is in the domain of the 
mapping sol and sol(M) = s. 



Propositional Satisfiability in Answer-Set Programming 145 



A search problem II is in the class NP-search if there is a nondeterministic 
Turing Machine TM such that (1) TM runs in polynomial time; (2) for every 
instance / S Djj, the set of strings left on the tape when accepting computations 
for I terminate is precisely the set of solutions Sn{I)- 

We now have the following theorem that determines the expressive power of 
the logic . Its proof is provided in the appendix. 

Theorem 1. A search problem U can be solved by a program if and only 

if n G NP-search. 

Decision problems can be viewed as special search problems. For the class of 
decision problems, Theorem H implies the following corollary (a counterpart to 
the result on the expressive power of DATALOG” 

Corollary 1. A decision problem U can be solved by a program if and 

only if n is in NP. 

4 Extending — The Logic PS~^ 

We will now discuss ways to enhance effectiveness of logic PS'^'^°‘ as a modeling 
formalism and propose ways to improve computational performance. When con- 
sidering the PS'^'^°' theories developed for the n-queens and vertex-cover prob- 
lems one observes that these theories could be simplified if the language of the 
logic contained direct means to model constraints such as: “exactly one 

element is selected” or “at most k elements are selected” . 

With this motivation, we extend the language of the logic PS™°‘ . We define 
a c-atom (cardinality atom) as an expression m{p{X, _,Y)}n, where m and n 
are non-negative integers, X and Y are tuples of variables and p is a program 
predicat^. 

The interpretation of a c-atom is that for every ground tuples x and y that 
can be substituted for X and T, at least m and at most n atoms from the set 

{p(a:, c, p): c is a constant appearing in the theory} 

are true. One of m and n may be missing from the expression. If m is missing, 
there is no lower-bound constraint on the number of atoms that are true. If 
m is missing, there is no upper-bound constraint on the number of atoms that 
are true. It is also possible to have more “underscore” symbols in c-atoms. In 
such case, when forming the set of atoms on which cardinality constraints are 
imposed, all possible ways to replace the “underscore” symbols by constants are 
used. 

An extended clause is a clause built of c-atoms. The notions of a program and 
theory are defined as in the case of the logic PS^'“°‘ . 

A theory in the extended syntax can be grounded, that is, represented as a 
set of propositional clauses, in a similar way as before. In particular, data and 

^ In our implementation, we support a somewhat more general form of c-atoms. 
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predefined predicates are treated in the same way and are subject to the same 
version of CWA that was used for the logic While grounding, c-atoms 

are interpreted as explained earlier. Grounding allows us to lift the semantics of 
propositional logic to the theories in the extended syntax. We call the resulting 
logic the extended logic and denote it by PS^ . 

In the logic PS^ we can encode the vertex cover problem in a more straight- 
forward and more concise way. Namely, the problem can be represented without 
the need for integers to label the vertices of an input graph! This new represen- 
tation is given by: 



D'.c = 


{vtx{v):v G 


V} U {edge{v, w): {u, ru} G E}, 


and P's = 




VC'l: 


invc{X) 


vtx{X) 


VC'2: 


{invc{fi'\k 




VC'3: 


edge{X, Y) 


invc{X) V invc(Y). 



Atoms invc{x) that are true in a model of the theory P'^) define 

a set of vertices that is a candidate for a vertex cover. (VC'2) guarantees that no 
more than k vertices are included. (VC'3) enforces the vertex-cover constraint. 

We close this section with an observation on the expressive power of the logic 
PS^ . Since it is a generalization of the logic PS‘^™°‘ , it can capture all problems 
that are in the class NP-search. On the other hand, the problem of computing 
models of a PS'^ theory with a fixed program part is an NP-search problem, it 
follows that the expressive power of the logics PS^ does not extend beyond the 
class NP-search. In other words, the logic PS^ also captures the class NP-search. 

5 Computing with PS~^ Theories 

To process PS'^ theories, one approach is to ground them into collections of 
propositional clauses. However, CNF representations of c-atoms may be quite 
large; the constraint “at most n atoms in the set {pi, . . . ,pk\ are true”, is cap- 
tured by clauses Pi^, - ■ ■ ,Pi„+i F, one for each (n-l- l)-element subset 

{Pii,---,Pin+i} of {pi,...,Pk}- 

Thus, we propose another approach. The idea is to develop an extension of 
propositional logic representing c-atoms directly. Let At be a set of proposi- 
tional variables. By a propositional c-atom we mean any expression of the form 
m{pi, . . . ,pfe}n, where m and n are non-negative integers and p\,...,pk are 
atoms in At (one of m and n may be missing). By an extended propositional 
clause we mean an expression of the form 

C= Ai A . . . A As ^ Hi V . . . V Ht, 

where all Ai and Bi are propositional c-atoms. 

Let M C At be a set of atoms. We say that M satisfies a generalized atom 
m{pi , . . . ,pk}n if 



m < |M n {pi, . . . ,pk}\ < n. 
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Further, M satisfies a generalized clause C if M satisfies at least one atom Bj or 
does not satisfy at least one atom Ai. We call the resulting logic the propositional 
logic PS^ . Clearly, M satisfies an atom l{p}l if and only if p G M. Thus, the 
propositional logic PS~^ extends the (clausal) propositional logic. 

Theories of the logic PS^ can be grounded in the extended propositional 
logic by generalizing the approach described in Section El We represent c-atoms 
as propositional c-atoms and avoid a blow-up in the size of the representation. 
The problem is that SAT checkers cannot now be used to resolve the satisfiability 
of the extended propositional logic as they are not designed to work with the 
extended syntax. 

It is clear, however, that the techniques developed in the area of SAT checkers 
can be extended to the propositional logic PS^ . We have developed a Davis- 
Putnam like procedure, aspps, that finds models of propositional PS~^ . We also 
developed a program psgrnd that accepts theories in the syntax of the logic PS^ 
and grounds them into propositional PS^ theories. Thus, the two programs 
together can be used as a processing mechanism for an answer-set programming 
system based on the logic PS~^ . The programs psgmd and aspps are available at 
http : / /www . cs . uky . edu/ ai /aspps/ 

In our experiments we considered the vertex-cover problem and several com- 
binatorial problems including n-queens problem, pigeonhole problem and the 
problem to compute Schur numbers. All our experiments were performed on a 
Pentium III 500MHz machine running linux. 

We were mostly interested in comparing the performance of our system ps- 
gmd/aspps with that of smodels. The reason is that both programs accept similar 
syntax and allow for very similar modeling of constraints. We also experimented 
with a satisfiability checker satz. 

In the case of vertex cover, for each n = 50, 60, 70 and 80, we randomly gen- 
erated 100 graphs with n vertices and 2n edges. For each graph G, we computed 
the minimum size kc for which the vertex cover can be found. We then tested 
aspps, smodels and satz on all the instances {G,kG)- The results represent the 
average execution times Encodings we used for testing aspps and smodels where 
based on the clauses (VC'l) - (VC'3). For satz we used encodings based on the 
clauses (VCl) - (VC6) (cardinality constraints cannot be handled by satz). 

A propositional CNF theory obtained by grounding the program (VCl) - 
(VC6), has 0(n^) atoms, 0{mn'^) clauses and its total size is also 0{mn^). For 
input instances we used in our experiments, these theories were of such large 
sizes (over one million rules in the case of graphs with 80 vertices) that satz 
did not terminate in the time we allocated (5 minutes). Thus, no times for satz 
are reported. On the other hand, since the propositional PS~^ theory obtained 
by grounding the PS~^ program (VC'l) - (VC'3) has only 6>(m -|- n) clauses (a 
few hundred clauses for graphs with 80 vertices) and its total size has the same 
asymptotic estimate. This is dramatically less than in the case of theories satz 
had to process. Both aspps and smodels performed very well, with aspps being 
about three times faster than smodels. The timing results are summarized in 
Table 1. 
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Table 1. Timing results (in seconds) for the vertex-cover problem. 



n 


50 


60 


70 


80 


aspps 


0.04 


0.22 


1.26 


6.45 


smodels 


0.12 


0.76 


4.14 


22.35 



For the n-queens problem, our solver performed exceptionally well. It scaled 
up much better than smodels both in the case when we were looking for one 
solution and when we wanted to compute all solutions. In particular, our program 
found a solution to the 36 queens problem in 0.97 sec. It also outperformed satz. 

Table 2. Timing results (in seconds) for the n-queen problem. 



# of queens 


18 


19 


20 


21 


22 


23 


aspps 


0.02 


0.02 


0.07 


0.07 


0.11 


0.12 


smodels 


2.35 


1.28 


13.25 


19.31 


167.1 


380.35 


satz 


1.16 


0.61 


4.35 


0.95 


28.64 


1.42 



The pigeonhole problem consists of showing that it is not possible to place p 
pigeons in h holes ii p > h. For this problem aspps showed the best performance 
— about three times faster than the other two solvers (all programs showed a 
similar rate of growth in the execution time). 

Table 3. Timing results (in seconds) for the pigeonhole problem 



{P,h) 


00 


(10,9) 


(11,10) 


(12,11) 


aspps 


0.59 


5.63 


60.08 


702.02 


smodels 


2.7 


21.56 


219.99 


2469.97 


satz 


1.87 


17.28 


178.20 


2044.42 



The Schur problem consists of placing n numbers 1, 2, . . . , n in A: bins so that 
the set of numbers assigned to a bin is not closed under sums. That is, for all 
numbers a;, y, z, 1 < a;, y, z < n, if x and y are in a bin b, then z is not in b (x 
and y need not be distinct). The Schur number S(k) is the maximum number n 
for which such a placement is still possible. 

We considered the problem of the existence of the placement for fc = 4 and 
values of n ranging from 40 to 45. For n < 44 all programs found a “Schur” 
placement. However, no “Schur” placement exists for n = 45 (and higher values 
of n) . All programs were able to establish the non-existence of solutions for n = 
45 (but the times grew significantly). Our results summarizing the performance 
of our system and smodels on the theories encoding the constraints of the problem 
are shown in Table 4. aspps and satz seem to performed better than smodels, 
with satz being slightly faster for values of n closer to the Schur number. 

In the case of the last three problems, it was possible to eliminate cardinality 
constraints without significant increase in the size of grounded theories. As a 
result, satz performed well. 
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Table 4. Timing results (in seconds) for the Schur-number problem. 



n 


40 


41 


42 


43 


44 


45 


aspps 


0.03 


0.03 


0.03 


0.03 


1.83 


54.5 


smodels 


0.3 


0.38 


0.32 


0.36 


35.8 


>1500 


satz 


0.21 


0.23 


0.24 


0.25 


0.96 


20.4 



6 Conclusions 

Our work demonstrates that propositional logic and its extensions can support 
answer-set programming systems in a way in which stable logic programming 
and disjunctive logic programming dc0. In the paper we described logic PS^ 
that can be used to this end. We presented an effective implementation of a 
grounder, psgrnd, and a solver, aspps, for processing theories in the logic PS~^ . 
Our experimental results are encouraging. Our system is competitive with smod- 
els, and in many cases outperforms it. It is also competitive with satisfiability 
solvers such as satz. 

The results of the paper show that programming front-ends for constraint 
satisfaction problems that support explicit coding of complex constraints facili- 
tate modeling and result in concise representations. They also show that solvers 
such as aspps that take advantage of those concise encodings and process high- 
level constraints directly, without compiling them to simpler representations, 
exhibit very good computational performance. These two aspects are impor- 
tant. Satisfiability checkers often cannot effectively solve problems simply due 
to the fact that encodings they have to work with are large. For instance, for 
the vertex-cover problem for graphs with 80 vertices and 160 edges, aspps has 
to deal with theories that consist of a few hundred of rules only. In the same 
time pure propositional encodings of the same problem contain over one million 
clauses — a factor that undoubtedly is behind much poorer performance of satz 
on this problem. 

Our work raises new questions. Further extensions of logic PS~^ are possible. 
For instance, constraints that impose other conditions on set cardinalities than 
those considered here (such as, the parity constraint) might be included. We 
will pursue this direction. Similarly, there is much room for improvement in the 
area of solvers for the propositional logic PS~^ . In particular, we will study local 
search algorithms as possible satisfiability solvers for propositional PS~^ theories. 

Finally, we note that the experimental results presented here are meant to 
show that aspps is competitive with other solvers and, we think, they demon- 
strate this. However, these results are still too fragmentary to provide basis for 
any conclusive comparison between the three solvers tested. Such a comparison is 
further complicated by the fact that the same problem may have several different 

® We point out, though, that stable logic programming and disjunctive logic pro- 
gramming directly support negation-as-failure and, consequently, yield more direct 
solutions to some knowledge representation problems such as, for example, the frame 
problem. 
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encodings with different computational properties. Developing the methodology 
for comparing solvers designed to work with different formal systems is a chal- 
lenging problem for builders of constraint solvers and declarative programming 
systems. 
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Appendix 

We will present here a sketch of a proof of our main result concerning the ex- 
pressive power of the logic The proof relies on some basic notions from 

logic programming (we refer the reader to l | Apt9()|l ,1oR4] for details). 

We restrict our discussion to function-free languages (the case relevant to 
our logic PS™°‘). Given a predicate language £ (as defined in Section 0, a logic 
program clause over this language is an expression r of the form 

r = Pif) (^l) ; ■ • ■ 5 Qm(j'm), tlOt (t^_|_l)) , . . . , not ) 

where p,qi, ... , qm+n C Pr, (we assume that p is not a predefined predicate), 
and t,ti, . . . are term tuples with the arity matching the arity of the 

corresponding predicate symbol. We call the atom p{t) the head of the rule r 
and denote it by h(r). For a rule r we also define 

B{r) = (^i(ti) A . . . qm(fim) A ~'Qm+l(^m-t-l) A ... A ~'qn(j'n) 

We will be interested in supported models of logic programs. Without loss of 
generality, we will restrict our attention to programs in the normal form. That 
is, we assume that (1) the head of each rule is of the form p{t), where t is a tuple 
of variables, and (2) if p appears in the head of two rules, the heads of these two 
rules are exactly the same (the same tuple of variables appear in both of them) 
IK;ia78^Ant90| . 

Let P be a program in the normal form. For each predicate symbol p G Pr{P), 
we define a formula cc{p) by: 

cc{p) = p{X) CA \J{3YrB{r)\ r G P' , h{r) = p{X)}, 

where X is a tuple of variables and Yr is the tuple of variables occurring in the 
body of r but not in the head of r (we exploit the normal form of P here). We 
define the completion of P, CC{P), by setting CC{P) = {cc{p)-.p S Pr}. 

The Clark’s completion is important as it allows us to characterize supported 
models of a logic program iCTn . Namely, we have the following result. 
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Theorem 2. Let P be a logic program. A set of ground atoms M is a supported 
model of P if and only if it is a Herbrand model of CC{P). 

We now have the following theorem. 

Theorem 3. Let P be a logic program in the normal form. Let Pr be the set of 
predicates appearing in P and let Pr' be the set of predicates of P that do not 
appear in the heads of rules in P. There is a theory T{P) such that for 

every set of ground atoms D over predicates from Pr' , a set of ground atoms M 
is a supported model of D U P if and only if M = M' fl HB{P) for some model 
M' of the PS'''"'' theory {D,T{P)). 

Proof: (Sketch) To define T{P), we consider the completion CC{P) of P. The 
idea is to take for T{P) an equivalent clausal representation of CC{P). 

We build such representation as follows. Let p be a predicate symbol in 
Pr \ Pr' . The completion CC{P) contains the formula 

cc(p) = p{X) \J{3YrB{r): r G P, h{r) = p{X)}, 

where X is a tuple of variables and Yr is the tuple of variables occurring in the 
body of r but not in the head of r. For each rule r G P such that p occurs in 
h{r), we introduce a new predicate symbol dr, of the same arity |X| + \Yr\. We 
define a theory T'{P) to consist of the following formulas (we recall that B{r) 
stands for the conjunction of the literals from the body of r): 

V'(r) = dr{X, Yr) ^ B{r), 

where p G Pr\ Pr' , r G P and p occurs in the head of r, and 

cc'(p) = p{X) ^ \J {3Yrdr{X,Yr)-.r G P,h{r) =p{X)}, 
where p G Pr\ Pr' . 

It is clear that the theory T'{P) is equivalent to CC{P) (modulo new ground 
atoms). That is, M C HB{P) is a model of CC{P) if and only if M = M' fl 
HB{P), for some model M' of T'{P). 

One can show that T'{P) can be rewritten (in polynomial time) into an equiv- 
alent clausal form, T{P). Consequently, T{P) is equivalent to CC{P) (modulo 
ground atoms dr{t)). It is now a routine task to verify that the theory T{P) 
satisfies all the requirements of the statement of the theorem. □ 

Using the terminology introduced here we will now prove Theorem Q from 
Section 0 

Theorem 4. A search problem LI can be solved by a finite PS™°‘ program if 
and only if II G NP-search. 

Proof: (Sketch) In jM H.OIj it is proved that every NP-search problem can be 
solved uniformly by a finite logic program under the supported-model seman- 
tics. Since the theory T(P) can be constructed in polynomial time, it follows 
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by Theorem 0 that every search problem in NP-search can be solved by a finite 
pgcwa Conversely, for every fixed program P, the problem of com- 
puting models of a theory (D,P) {D is the input) is clearly in the class 

NP-search. Thus, only search problem in the class NP-search can be solved by 
finite PS™°“ programs. Hence, the assertion follows. □ 



Prediction of Regular Search Tree Growth by 
Spectral Analysis 



Stefan Edelkamp 



Institut fiir Informatik 
Georges-Kohler-Allee, Gebaude 51 
79110 Freiburg 

edelkamp@informatik.uni-freiburg.de 



Abstract. The time complexity analysis of the IDA* algorithm has 
shown that predicting the growth of the search tree essentially relies 
on only two criteria: The number of nodes in the brute-force search tree 
for a given depth and the equilibrium distribution of the heuristic es- 
timate. Since the latter can be approximated by random sampling, we 
accurately predict the number of nodes in the brute-force search tree for 
large depth in closed form by analyzing the spectrum of the problem 
graph or one of its factorization. 

We further derive that the asymptotic brute-force branching factor is in 
fact the spectral radius of the problem graph and exemplify our consid- 
erations in the domain of the (n^ — 1)-Puzzle. 



1 Introduction 

Heuristic search is essential to AI, since it allows very large problem spaces to be 
traversed with a considerably small number of node expansions. Nevertheless, 
storing this number of nodes in memory, as required in the A* algorithm |^, 
often exceeds the resources available. This is bypassed in an iterative deepening 
version of A*, IDA* for short, that searches the tree expansion of the original 
state graph instead of the graph itself. IDA* p]j applies bounded depth-first 
traversals with an increasing threshold on A*’s node evaluation function. The 
tree expansion may contain several duplicate nodes such that low memory con- 
sumption is counterbalanced with a considerably high overhead in time. 

Fortunately, due to simple search tree pruning rules and expressive heuristic 
estimates to direct the search process, duplicates in regular search spaces are 
rare such that IDA* has been very successfully applied to solve solitaire games 
like the — 1) Puzzle mnsm and Rubik’s Cube HH. 

Korf, Reid and Edelkamp m have analyzed the IDA* algorithm to predict 
the search performance of IDA* in the number of node expansions for a specific 
problem. The main result is that assuming consistencjQ of the integral heuristic 

^ Gonsistent heuristic estimates satisfy h{v) — h{u) -I- 1 > 0 for each edge (u, v) in the 
underlying problem graph. They yield monotone node evaluations f{u) = g{u) + 
h{u) on generating paths with length g(u). Admissible heuristics are lower bound 
estimates that underestimate the goal distance for each state. Consistent estimates 
are admissible. 
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estimate in the limit of large c, the expected total number of node expansions 
with cost threshold c in one iteration of IDA* is equal to 



— d), 

d—0 

where is the number of nodes in the brute-force search tree with depth 
d and P is the equilibrium distribution defined as the probability distribution of 
heuristic values in the limit of large depth. More precisely, P{h) is the probability 
that a randomly and uniformly chosen node of a given depth has a heuristic value 
less than or equal to h. In practice the equilibrium distribution for admissible 
heuristic functions will be approximated by random sampling |E| ; a represen- 
tative sample of the problem space is drawn and classified according to the 
integral heuristic evaluation function. The value for large depths d without 
necessarily exploring the search tree, can be approximated with the asymptotic 
brute-force branching factor; the number of nodes at one depth divided by the 
number of nodes in the next shallower depth, in the limit as the depth goes 
to infinity. The asymptotic heuristic branching factor is defined analogously on 
search tree levels for two occurring values on the node evaluation function /. In 
some domains we observe anomalies in the limiting behavior of the asymptotic 
branching factors, e.g., in the (n^ — 1)-Puzzle and odd values of n it alternates 
between two different values 0. 

The observation that a consistent heuristic estimate h affects the relative 
depth to a goal instead of the branching itself is supported by the fact that 
IDA*’s exploration is equivalent to undirected iterative deepening exploration 
in a re-weighted problem graph with costs 1 -I- h{v) — h{u) for all edges (u,v). 
The new node evaluation f'{uj) of node Uj on path p = (s = ui, ... ,Ut = t) 
equals (1 + h{ui+i) — h{ui)) and telescopes to the old merit f{uj) minus 

h{s). Therefore, the heuristic is best understood as a bonus to the search depth. 
Moreover, since we have only altered edge weights, it is not surprising that for 
bounded heuristic estimates and large depth the asymptotic heuristic branching 
factor equals the asymptotic brute-force branching factor. 

Our main result in this paper is that in undirected problem graphs the value 
of the number of nodes in depth d of the brute-force search can be computed 
effectively by analyzing the spectrum of the adjacency matrix for the problem 
graph. The analysis requires some results of linear algebra and an algorithm 
of applied mathematics. Since the problem graph is considered to be large for 
regular search spaces we show how to factorize the problem graph through an 
equivalence relation of same branching behavior. We take the (n^ — 1)-Puzzle 
as the running example, discuss the generality of the results from various points 
of view: other problem domains, general, especially undirected graph structures, 
and predecessor pruning. Finally, we give concluding remarks and shed light on 
future research options. 
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2 Linear Algebra Basics 

Linear Mappings and Bases. A mapping f : V ^ W, with V, W being vector 
spaces over the field K (e.g. the set of real or the set of complex numbers) is 
linear, if f{\v + piw) = A/(u) + p,f{w) for all v,w € V and all X, p, € K . A basis 
of a vector space is a linear independent set of vectors that spans V. If the 
basis is finite, its cardinality defines the dimension dim{V) of the vector space 
V , otherwise the dimension is said to be infinite. 

Matrices and Basis- Transformations. Linear mappings of vector spaces of finite 
dimension can be represented as matrices, since there is an isomorphism that 
maps the set of all (m x n) matrices to the set of all linear mappings from 
V to W according to their respectively fixed bases, where dim(V) = n and 
dim{W) = m. Usually, V equals W and in this case the linear mapping / is 
called endomorphism. A basis-transformation from basis A to B m. the vector 
space V can be represented by a transformation matrix which is the inverse 
of Very often, A is the canonical basis. Computing the inverse C~^ of a 
matrix C can be achieved by elementary row transformations, that convert the 
(n X 2n) matrix \C \ I] into [/ | C~^], with I being the identity matrix. 

Similarity and Normal Forms. Two matrices A and B are similar, if there is 
a matrix C with B = CAC~^. This is equivalent to the fact that there is an 
endomorphism f of V and two bases A and B with matrix A representing / ac- 
cording to A and B representing / according to B. Similarity is an equivalence 
relation and one main problem in linear algebra is to derive a concise representa- 
tive in the equivalence class of similar matrices, the normal form. A very simple 
form is the diagonal shape with non-zero values Ai,...,A„ only on the main 
diagonal. In this case, a matrix B is called diagonizable and can be written as 
B = C ■ diag(Ai, . . . , A„) • C~^. Unfortunately, not all matrices are diagonizable, 
especially when the linear mapping is defined on the set of real numbers. Even 
if the vector space defining field is the set of complex numbers, only tridiagoniz- 
ability can be granted, in which matrix A may have non-zero components above 
the main diagonal. Further simplifications lead to the so-called Jordan normal 
form. 

Eigenvalues and Eigenspaces. An endomorphism / of a vector space V over the 
field K contains an eigenvalue A G K, if there is a non-trivial vector v G V, 
with f{v) = Xv. Any such non-trivial vector v G V with f(v) = Xv is called 
eigenvector. If there is a basis B of eigenvectors, then the matrix representa- 
tion according to B has a diagonal shape. In this case / is also called diag- 
onizable. It can be shown that the eigenvalues are roots of the characteristic 
equation Pf{X) = det(A — XI) = 0, where the determinant det(A) is defined as 
ErreSn rii<j(f^O’)-^(*))/(j-*))-ai<T(i)-- ■ --anain) with S'„ being the set of all n- 
permutations. If the polynomial Pf{X) factorizes, i.e. Pf{X) = const-J)[iLi(A— A*), 
which is the case for matrices of complex numbers, the corresponding eigenspaces 
Ef{Xi) have to be computed. If then the number of occurring linear terms (A— A^) 
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in the algebraic multiplicity of A^, equals the dimension of Ef{Xi), the 

geometric multiplicity of A^, then A is indeed diagonizable. 

3 Partitioning the Search Space 

The {n? — 1)-Puzzle is a sliding tile toy problem. It consists of (n^ — 1) numbered 
tiles that can be slid into a single empty position, called the blank. The goal is to 
rearrange the tiles such that a certain goal position is reached. Figure E depicts 
possible end configurations of well-known instances to the {n? — 1)-Puzzle: For 
n = 3 we get the Eight-Puzzle, for n = 4 the Fifteen-Puzzle and for n = 
5, the Twenty-Four-Puzzle is met. The state spaces for these problems grow 
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Fig. 1. The Eight-, Fifteen- and Twenty-Four-Pnzzles. 



exponentially. The exact number of reachable states (independent of the initial 
one) is (n^)!/2 which resolves to approximately 10^ states for the Eight-Puzzle, 
10^^ states in the Fifteen-Puzzle and 10^^ states in the Twenty-Four-Puzzle. 

We partition the search space S in classes S\, ... ,Sk, collecting states into 
groups with same branching behavior. In other words we devise an equivalence 
relation that partitions the state space into equivalence classes: two states are 
equivalent if their long term branching behavior coincides. All states in one 
equivalence class 5'i, i £ {1, . . . , A:}, necessarily have the same node branching 
factor, defined as the number of children a node has in the brute-force search 
tree. 

For the example of the {v? — 1)-Puzzle a partition is given by the following 
relation: two states are equivalent if the blank is at the same absolute position. 
Obviously the subtrees of such nodes are isomorphic, since the branching be- 
havior of equivalent states has to be the same. A further reduction of the search 
tree is established by partitioning the search space with respect to symmetry. 
For the {in? — 1) Puzzle we establish three branching types: corner or c-nodes 
with node branching factor 2, side or s-nodes with node branching factor 3, and 
middle or m-nodes with node branching factor 4. However, does the long time 
node branching behavior depend on these node types only? In the Eight- and 
Fifteen-Puzzles this is the case, since for symmetry reasons all c, s and m nodes 
generate the same subtree structure. For the Twenty-Four-Puzzle, however, the 
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Fig. 2. Equivalence Graph for the Eight-, Fifteen- and Twenty-Four-Puzzles. 



search tree of two side or two middle states may differ. For this case we need six 
classes with a blank at position 1,2, 3, 7, 8, and 13 according to the tile labeling 
in Figure 13 In the general case the number of different node branching classes 
in the (n^ — 1) Puzzle is 



r ^/^1 / r \ 

lj = K2l(rn/2l-l)/2. 



This still compares well to a partition according to the equivalent classes 
in the first factorization (savings of a factor of about eight) and of course to the 
(n^)!/2 states in the overall search space (exponential savings). 



4 Equivalence Graph Structure 

Utilizing this partition technique we define the weighted equivalence graph G = 
{V,E,w) as follows. The set of nodes V equals the set of equivalence classes 
and an edge e from class Si € V to Sj G V with weight w{e) is drawn, if every 
state in Si leads to w states in class Sj . Obviously, the sum of all outgoing edges 
equals the node branching factor. Let Aq be the adjacency matrix with respect 
to the equivalence graph G. Since the explorations in G and G span the same 
search-tree structure the search tree growth will be the same. 

A generator matrix P for the population of nodes according to the given 
equivalence relation is defined by P = AA. More precisely, Pjy = Z if a node of 
type Z in a given level leads to I nodes of type j in the next level. We immediately 
infer that with being the vector of nodes in depth d of 

the search tree. If || • ||i denotes the vector norm ||a:||i = |a;i| -I- . . . -I- |a;fc| then 
the number of nodes in depth d is equal to | | |i. 

The asymptotic branching factor b (if it exists) is defined as the limit of 
for increasing d and equals the weighted product of the node fre- 
quencies b = bifi, where fi is the fraction of nodes of class i with respect 
to the total number of nodes. As we will see, we can compute the branching 
factor analytically without actually determining node frequency values. 
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The first observation is that in case of convergence the asymptotic branch- 
ing factor is not only met in the the overall search tree expansion but in ev- 
ery equivalence class. Since all frequencies of nodes converge we have that 
b = limd^oo with being the number of nodes of class i in 

depth d, i G fc}. In other words, if the ratio of the cardinality of one 

equivalence class and the overall search space size settles and the search space 
size grows with factor b, then the equivalence class size itself grows with factor 
b. 

We represent the fractions /i as a distribution vector F. We first assume that 
this vector converges in the limit of large depth. The considerations for an ana- 
lytical solution to the branching factor problem result in the equations bF = FP, 
where b is the asymptotic branching factor. In addition, we have the equation 
that the total of all node frequencies is one. The underlying mathematical is- 
sue is an eigenvalue problem. Transforming bF = PF leads to 0 = (P — bI)F 
for the identity matrix I. The solutions for b are the roots of the characteristic 
equation det(P — bl) = 0 where det is the determinant of the matrix. Since 
det(P — bl) = det(P^ — bl) the transposition of the equivalence graph matrix 
Aq preserves the value of b. In case of the Eight-Puzzle det(P — bl) equals 

/0-b 2 0 \ 

det 2 0-6 1 =0. 

\ 0 4 0 - 6/ 

This equation is equivalent to 6(4 — 6^) -I- 46 = 0, yielding the following three 
solutions — \/8 = —2.828427124, 0, -\/8 = 2.828427124. Experimental results 
show that the branching factor alternates every two depth values between 3 and 
8/3 = 2.666666666. Since -\/8 is the geometric mean of 3 and 8/3 the value -\/8 is 
the proper choice for the asymptotic branching factor 6 of the brute-force search 
tree. 

For the case of the Fifteen-Puzzle we have to calculate 

/0 - 6 2 0 \ 

det I 1 1 — 6 1 ) = 

\ 0 2 2 - 6/ 

which simplifies to (1 — 6) (6 — 2)6-1- 46 — 4 = 0. The solution to this equation 
are 1, 1 -b 75 = 3.236067978, and 1 - 75 = -1.236067978. The value 1 -b 75 
matches experimental data for the asymptotic branching factor. 

For the Twenty-Four-Puzzle we have to solve 

/0 - 6 2 0 0 0 0 \ 

1 0-6 1100 
0 20-60 1 0 
0 2 0 0 - 6 2 0 

0012 0-6 1 
\ 0 0 0 0 4 0 - 6/ 
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The six eigenvalues are 0, 0, -\/3 = 1.732050808, — -\/3 = —1.732050808, 
\/l2 = 3.464101616, and — = —3.464101616. Experiments show that for 
large depth the branching factor oscillates and that the geometric mean is 
3.464101616. 

We conclude that the asymptotic branching factor in the example problems 
is the largest eigenvalue of the adjacency matrix for the equivalence graph and 
that we observe anomalies if the largest eigenvalue has a negative counterpart 
of the same absolute value. In the following we will analyze the structure of the 
eigenvalue problem to show why this is the case. 

5 Exact Prediction of Search Tree Size 

The equation can be unrolled to We briefly 

sketch how to compute P‘^ for large d. We have seen that P is diagonizable, if 
there exists a invertible matrix C and a diagonal matrix Q with P = CQC~^. 
This simplifies the calculation of P’^, since we have P’^ = CQ’^C~^ (the remaining 
terms C~^C cancel). By the diagonal shape of Q the value of is obtained 
by taking the matrix elements qi^i to the power of d. These elements are the 
eigenvalues of P. This connection is not surprising, since in case of convergence 
of the vector of node frequencies F we have seen that the branching factor itself 
is a solution to the eigenvalue problem PF = bF. We conclude that in case 
of diagonizability we can exactly predict the number of nodes of depth d by 
determining the set of eigenvalues of P. 

In the example of the Eight-Puzzle the eigenvectors for the eigenvalues — -\/8, 
0, and -\/8 are (2, — \/8, 1)^, (—2, 0, 1)^, and (2, -\/8, 1)^, respectively. Therefore, 
the basis-transformation matrix C is given by 

/ 2 -2 2 \ 

C= -78 0 78 

V 1 1 1 / 

with the following inverse 

/ 2 -T84\ 

C~^ = 1/16 -2 0 8 . 

V 2 78 4/ 

With Q = diag{—V&, 0, 78) we have C~^C = I and C~^PC = Q. Therefore, 
calculating for d > 1 corresponds to where 

Q<^ = diag((— TS)'^, 0, (78)*^). Hence, equals to 

/ 2 -2 2 \ /(-78)‘^0 0 \ / 2 -784\ /l\ 

1/16 -78 078 0 0 0 -2 0 8 0 

V 1 1 1 / V 0 0 (78)V V 2 78 4/ \oJ 

which resolves to 

ivW = 1/16 (478‘^((-l)''-h 1),278'^’^\(-1)‘^+^ -k l),278'^((-l)^-k 1))"^. 




Prediction of Regular Search Tree Growth by Spectral Analysis 161 



The exact formula for and small values of d validates the observed search 

tree growth: = (0,2,0)^, = (4,0,2)^, = (0,16,0)’^, = 

(32,0,16)^, etc. 

The closed form for explicitly states that the asymptotic branching 

factor for the Eight Puzzle is \/8. Moreover, the odd-even effect for branching 
in that puzzle is established by the factor (— + 1, which cancels for an odd 
value of d. Nevertheless, solving the characteristic equation and establishing the 
basis of eigenvectors by hand is tedious work. Fortunately, the application of 
symbolic mathematical tools such as Maple and Mathematica help to perform 
the calculations in larger systems. 

For the Fifteen-Puzzle the basis-transformation matrix C and its inverse 
are 

/I _l 1 \ 

C= 1-V5 -1 1 + V5 

\3/2-l/2y5 1 3/2-bl/2V5/ 

and 

/ 1/50 (5 -b 3^5) 75 -1/50(5-^75)75 l/5\ 

C~^ = -2/5 -1/5 2/5 . 

Vl/50 (-5 -b 3 75) 75 -1/50 (-5 -b 7s) 7S 1/5/ 

The vector of node counts is 

1/50 (1 - 75)"* (5 -b 3 75) 75 -b 2/5-b 
1/50 (1 -b 75)"* (-5 -b 3 75) 75 

1/50 (1 - 75) (1 - 75)"* (5 -b 3 75) 75 -b 2/5-b 
1/50 (l-b7s) (l-b75)"*(-5-b3 75)75 

1/50 (3/2-1/2 75) (1 - 75)"* (5 -b 3 75) 75 -2/5-b 
1/50 (3/2 -b 1/2 75) (1 -b 75)"* (-5 -b 3 7s) 75 

such that the exact total number of nodes in depth d is 

1/50 (7/2-3/275) (l-75)‘^(5-b3 75)75-b2/5-b 

1/50 (7/2 -b 3/2 75) (1 -b 75) (-5 -b 3 75) 75 

The number of corner nodes (1,0,2,2,10,26,90,. . . ), the number of side nodes 
(0,2,2,10,26,90,282,...) and the number of middle nodes (0,0,6,22,70,230,...) 
grow as expected. The largest eigenvalue 1 -b 7S dominates the growth of the 
search tree in the limit for large d. 

In the Twenty-Four-Puzzle the value equals 
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' 1/36 (-2 + 2/9 + 2/9 (Ts)"' + 1/36 (2 ^3)'' ^ 

-1/18 ^3 (-2 V3)'^ - 2/9 73 (-73)“^ + 2/9 73 (73)'^ + 1/18 73 (2 73)"* 
1/18 (-2 73)“^ + 1/9 (-73)"* +1/9 (73)"* + 1/18 (2 73)"* 

1/12 (-273)"*+ 1/12 (273)"* 

-1/18 73 (-2 73)"* + 1/9 73 (-73)"* - 1/9 73 (73)"* + 1/18 73 (2 73)"* 

^ 1/36 (-2 73) "*-1/9 (-73) "*-1/9 (73) "* + 1/36 (273)"* ) 

for the following total of nodes in depth d 

7"*) = 1/36 ( j -4 :Vtj +1/9 (2-73) (^-73)^^ + 

1/9 (2 + 73) (T3)‘^ + 1/36 (7 + 473) (2 73)^ 

The value for small d validates that the total number of nodes increases as 
expected (2,6,18,60,198,684,. .. )• Once again the vector of the largest absolute 
value determines the search tree growth. 

If the size of the system is large, the exact value of has to be approxi- 
mated. One option to bypass the intense calculations for determinants of large 
matrices and roots of high-degree polynomials is to compute the asymptotic 
branching factor b. The number of nodes in the brute-force search tree is then 
approximated by nO) « b'^. 

6 Approximate Prediction of Search Tree Size 

The matrix denotation for calculating the population of nodes according to the 
given equivalence relation implies with being the vector of 

equivalent class sizes. The asymptotic branching factor b is given by the limit of 
which equals in any component i G {1, . . . , k}. 

Evaluating 7v)"*V-^i"* for increasing depth d is exactly what is considered in 
the algorithm of van Mises for approximating the largest eigenvalue (in absolute 
terms) of P. The algorithm is also referred to as the power iteration method. 

As a precondition, the algorithm requires that P be diagonizable. This implies 
that we have n different eigenvalues Ai, . . . , A„ and each eigenvalue \ with multi- 
plicity of ai has ai linear independent eigenvectors. Without loss of generality, we 
assume that the eigenvalues are given in decreasing order I All > IA2I > ... > |Afc|. 
The algorithm further requires that the start vector have a representation 
in the basis of eigenvectors in which no coefficient according to Ai is trivial. 

We distinguish the following two cases: |Ai| > IA2I > ... > |Afe| and |Ai| = 
IA2I > ... > |Afc|. In the first case we obtain that (independent of the choice of 
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j G the value of limd->.oo equals |Ai|. Similarly, in the 

second case lim^-^oo is in fact Af. The cases |Ai| = ... = |Aj| > 

. . . > |Afc| for ? > 2 are dealt with analogously. The outcome of the algorithm 
and therefore the limit in the number of nodes in layers with difference I is |Ai|*, 
so that once more the geometric mean turns out to be |Ai|. 

We indicate the proof of the first case only. Diagonizability implies a basis of 
eigenvectors b\...,bk- Due to |Ai| > IA2I > ■■• > |A„| the quotient of 
converges to zero for large values of d. If the initial vector with respect to 
the eigenbasis is given as xibi + X262 + . . . + Xkbk applying P‘^ yields xiP‘^bi + 
X 2 P’^b 2 + . . . + XkP'^hk by linearity of P, which further reduces to XibiXf + 
X 2 X 2 b 2 + - ■ . + Xf^Xkbk by the definition of eigenvalues and eigenvectors. The term 
xi^iAf will dominate the sum for increasing values of d. Factorizing Xf in the 
numerator and Alf~^ in denominator of the quotient of results in an 

equation of the form xi6iAi + i? where limd_>oo R is bounded by a constant, since 
except of the leading term xib\Xi both numerator and denominator in R only 
involve expressions of the form 0(|Ai/Ai|'^). Therefore, to find the asymptotic 
branching factor analytically, it suffices to determine the set of eigenvalues of P 
and to take the largest one. This corresponds to the results of the asymptotic 
branching factors in the {n? — 1)-Puzzles. 

In the Eight-Puzzle the ratio is equal to 8 for d > 3 and, there- 

fore, approximates Af. The value /\/S alternates between 3/4 and 
Hence, \/8 approximates the search tree growth. 

For the Fifteen-Puzzle for increasing depth d the value equals 1, 

3, 13/5, 45/13, 47/15, 461/141, 1485/461, 4813/1485, 15565/4813, 50381/15565, 
163021/50381, 527565/163021 = 3.236178161, etc., a sequence approximating 
1+V5 = 3.236067978. Moreover, the ratio of and (l-|--\/5)‘^ quickly converges 
to 1/50 (7/2 -b 3/2 \/5) (-5 -b 3 Vs) Vs = .5236067984. 

In the Twenty-Four-Puzzle the ratio converges to 12 starting 

with the sequence 6, 9, 11, 129/11, 513/43, 683/57, 8193/683, 32769/2731, 
43691/3641 = 11.99972535, etc. The quotient larger depth al- 

ternates between .3888888889 and .3849001795 and is therefore bounded by a 
small constant. 

If n is even - as in the Fifteen-Puzzle - the largest eigenvalue is unique and 
if n is odd - as in the Eight- and in the Twenty-Four-Puzzle - we find two 
eigenvalues with the same absolute value verifying that every two depths the 
node sizes will asymptotically increase by the square of these values. 

7 Generalizing the Approach 

Iterating the algorithm with ||fV^'^^||i/||iV(‘^“^Vi instead of shows 

that the convergence conditions according to G and G are equivalent. This is 
important, since other graph properties may alter, e.g. symmetry of Aq is not 
inherited by A^. Therefore, we concentrate on diagonizability results of Aq, 
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which are easier to obtain. The Theorem of Sehur states that symmetric matrices 
are indeed diagonizable. Moreover, the eigenvalues are real and the matrix to 
perform the basis transformation has the eigenvectors in its columns. 

For the {n? — 1)-Puzzle we are done. Since G is undirected, Aq is indeed sym- 
metric. In the spectrum of Aq power iteration either obtains a unique branching 
factor b = |Ai| or a branching factor of for every two iterations. Therefore, 
the branching factor is the speetral radius p = |Ai|. 



7.1 Other Problem Spaces 

Since the search tree is often exponentially larger than the problem graph we have 
reduced the prediction of the search tree growth to the spectral analysis of the 
explicit adjacency representation of the graph. As long as this graph is available, 
accurate and approximate predictions for the brute-force and subsequently for 
the heuristic search tree growth can be computed. 

However, the calculations for large implicitly given graphs are involved such 
that reduction of the analysis to a smaller structure is desirable. For the {nf — 1)- 
Puzzle we proposed a compression to a few branching classes. The application 
of equivalence class reduction to exactly predict the search tree growth relies on 
the regular structure of the problem space. This technique is available as long 
as the same branching behavior for different states is given. 

For Rubik’s Cube without predecessor elimination equals since all 
nodes in the search tree span a complete 18-ary subtree. With predecessor elimi- 
nation the node branching factor reduces to 15, since for each of the three twists 
single eloekwise, double cloekwise, and eountercloekwise there is a remainder of 
five sides front, baek, right, left, up, and down that are available. If we further 
restrict rotation of opposite sides to exactly one order we get the transition ma- 
trix ((66), (96)), where the first class is the set of primary nodes with branching 
factor 15, and the second class is the class of secondary nodes with branching 
factor 12. The eigenvalues are 6 -I- 3-\/6 and 6 — 3-\/6 and the value equals 
1/2 (6-1-3 -\/6) -1-1/2 (6 — 3 -\/6) . For small values of d experimental data as 
given in HH matches this analytical study. The observed asymptotic branching 
factor is 6 -I- 3-\/6 = 13.34846923 as expected. 

Extending the work to problem domains like the PSPACE-complete Sokoban 
problem fP is challenging. It is difficult to derive an accurate prediction, since 
the branching behavior of the tree includes almost all state facets. Therefore, a 
more complicated search model has to be devised to derive exact or approximate 
search tree prediction in this domain. As Andreas Junghanns has coined in his 
Ph.D. dissertation 0, the impact of the search tree node prediction formula 
— d) has still to be shown. In the other PSPACE-complete slid- 
ing block game Atomix p/l6j simplification based on branching equivalences do 
apply and yield savings that are exponential in the number of atoms, but this 
void labeling scheme still results in an intractable size of the equivalence graph 
structure. Only very small games can be analyzed by this method. 
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7.2 Pruning 

When incorporating pruning to the exploration process, symmetry of the under- 
lying graph structure may be affected. Once again we consider the Eight-Puzzle. 
The adjacency matrix for predecessor elimination now consists of four 

classes: cs, sc, me and cm, where the class ij indicates that the predecessor of a 
j-node in the search tree is an i node. 

0 10 0 
10 0 1 
2 0 0 0 
0 0 3 0 

In this case we cannot infer diagonizability according to the set of real numbers. 
Fortunately, we know that the branching factor is a positive real value since 
the iteration process is real. Therefore, we may perform all calculation to pre- 
dict the search tree growth with complex numbers, for which the characteristic 
polynomial factorizes. The branching factor and the search tree growth can be 
calculated analytically and the iteration process eventually converges. 

In the example, the set of (complex) eigenvalues is z-\/2, —1^/2, -\/3, and —\fZ. 
Therefore, the asymptotic branching factor is -\/3- The vector is equal to 

1/5 (zV2)'^-bl/5 (-zV2)‘^-b3/10 (73)"' -b 3/10 
-1/10 zV2 -b 1 / 10 z72 (-zV^)'^ -b 1/10^3 (V3)'' - 1/10 ^3 (-^3)'^ 

3/20zV2 {is/2)'^ - 3/20zV2 (-zv^)'^ -b 1/10 ^3 - 1/10 VS 

-1/10 (zV2)'^-l/10 (-zV2)''-bl/10 (y3)''-bl/10 (-V3)'^ 

Finally, the total number of nodes in depth d is 

= 1/5 (l/2-b l/dtv^) {i^/2^ (l/2-l/4iy2) + 

1/10 (4-b2 73) ( 73 )% 1/10 ( 4 - 2 ^ 3 ) 

For small values of d the value zz^'^^ equals 1, 2, 4, 8, 10, 20, 34, 68, 94, 188 etc. 

7.3 Non-diagonizability 

Since we assumed diagonizability, the eigenspaces L(Xi) according to the values 
Xi have full rank a^. In general this is not true. Not all matrices are diagonali- 
zable. In this case the best thing one can do is to transform the matrix into 
Jordan Form which has blocks on the diagonal, each block being r x r, with 
the eigenvalue on the diagonal, I’s above the diagonal and O’s everywhere else. 
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More precisely, a matrix A has Jordan Form J for an invertible matrix T, if 
J = T~^AT consists of so-called Jordan-blocks J\, . . . , Jm- One Jordan-block 
has an eigenvalue on the main diagonal and Is on the diagonal above. Therefore, 
T gives a basis of eigenvectors and so-called main vectors. Each Jordan-block 
Ji of dimension ji corresponds to one eigenvector t\ and ji — 1 main vectors 
t 2 , ■ ■ ■ Aji with (A - XiI)to = 0 and {A - Xil)tm = tm-i, m = 2, . . . ,j/. Using 
the Jordan basis one can devise similar to the case above. 

7.4 Start Vector 

The second subtlety arises even if the matrix is diagonalizable. We are interested 
in determining the behavior of for large d, where P is an n x n matrix and 

is an n X 1 vector. Suppose that P is diagonalizable, which means that there 
is a basis of eigenvectors. Hence, can be written as a sum of eigenvectors: 
iV(o) = v\ + V 2 + V's + . . . + Vn where vi is an eigenvector with eigenvalue Xi . It 
follows that = Xfvi + X 2 V 2 + X^V 3 + ... + Xfi^Vn So the term with the largest 

corresponding eigenvalue will dominate for large d, provided that the eigenvector 
is non-zero. It may happen that the initial vector v has component of zero in the 
eigenspace of the largest eigenvalue. In general, the algorithm finds the largest 
eigenvalue in which the corresponding component is non-zero. 

Fortunately, this observation is more theoretical in nature. In the iteration 
process this case is very rarely fulfilled. Rounding errors will soon or later lead to 
non-zero components. Moreover, to determine the asymptotic branching factor 
we have several initial states to choose from such that at least one has to yield 
non-zero coefficients. 

8 Previous Work 

This paper extends the work of Edelkamp and Korf [2| that already derived 
the asymptotic branching factors of the sliding-tile puzzles and Rubik’s Cube. 
However, their approach lack sufficient convergence conditions. We established 
the criterion of diagoniz ability of the adjacency graph matrix of the problem 
graph that emerges of the algorithm of van Mises and showed that this criterion is 
fulfilled in undirected graphs by the Theorem of Schur. The (n^ — 1)-Puzzles and 
Rubik’s Cube are chosen to illustrate the techniques, since they are inherently 
difficult to solve and often considered in case studies. 

The set of recurrence relations in | 2 | also showed that the numbers of nodes 
at various depths can be calculated in time linear to the product of the number of 
node classes and the depth of search tree by numerically iterating the recurrence 
relations. In contrast to this finding, the current paper resolves the problem of 
how to compute a closed form for the number of nodes. Last but not least, the 
given mathematical formalization of equivalence classification, diagonalization 
and power iteration builds a bridge for more powerful results in applying known 
mathematical theorems. At least in theory, generality to different problem spaces 
is given, since this approach applies to any problem graph with a diagonizable 
matrix and probably to more than that. 
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9 Conclusion and Discussion 

In the paper we have improved the prediction of the number of node expansions 
in IDA* by an exact derivation of the number of nodes in the brute-force search 
tree. We have resolved the question of convergence to explain anomalies in of 
the asymptotic branching factor. The asymptotic branching factor is the spectral 
radius of the successor generation matrix and can be computed with the power 
iteration method. The approach extends to further regular problem spaces and 
can cope with simple pruning rules. The main result is that diagonizability is 
granted in undirected problem graphs, such that exact and approximate calcu- 
lation of the brute-force search-tree are mathematically sound. The technique 
for establishing a closed form is not standard, and it is hard to suggest other 
methodologies to actually solve the set of recurrence relations. 

Moreover, given the adjacency matrix P of an undirected graph by analyzing 
Nf’'^ /N'f' and /N^'^ fc > 1, of the equation gives the 

(mean) asymptotic branching factor. This is in fact the algorithm of van Mises to 
determine the largest eigenvalue of P for whose applicability we have to test if P 
is diagonizable. The paper closes the small gap in literature to accurately predict 
search tree growth in closed form and to compute the branching factor both 
analytically and numerically without relying on strong experimental assumption 
on the convergence. 

Since for practical problems in which IDA* applies it is very unlikely that 
the entire graph structure can be kept in main memory, the approach helps 
only if some reduction of the branching behavior with respect to equivalence 
classes can be obtained. Therefore, the analysis is limited to the cases where the 
the successor generator matrix of the original or the adjacency graph structure 
can be build. If not, abstractions to the graph structure have to be found that 
preserve or approximate information of the branching behavior. 

All analyses given in this or precursory papers on search tree prediction do not 
include the application of transposition tables, in which visited states together 
with their best encountered state merits (path length plus heuristic estimate) 
are kept. This in fact is also a challenge for analysts. One option is the prediction 
of the search tree growth of IDA* with respect to bit-state hashing, which turns 
out to be an improvement to transposition tables in single-agent games 0 and 
protocol verification P| . For this model of partial search first results on coverage 
prediction have been found 0. 

Exact calculation of the brute-force search tree raises the question if the 
other source of uncertainty, namely the heuristic equilibrium distribution, can 
also be eliminated. As said, the equilibrium distribution of the estimate can be 
obtained by random sampling. However, in some cases of regular search trees 
exact values can be produced. If the estimate is given with respect to a pattern 
database storing pairs of the form (estimated value, state pattern) by analyzing 
the pattern database, a histogram of heuristic values can computed: we deter- 
mine the number of states that satisfy a pattern with a total to be computed 
for each integral heuristic value in a predefined range. For consistent heuristics 
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this range will be bounded by the heuristic estimate of the start state and the 
optimal solution length. 

At the very far end of this research line there are precise or approximate 
predictions for the growth of A*’s and IDA*’s search efforts according to various 
kinds of heuristics, node caching strategies and problem domains. This implies 
an alternative way of defining heuristics themselves: ranking successor nodes 
according to the expected growth of the resulting search tree. 
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Abstract. Having to cope with memory limitations is an ubiquitous issue in 
heuristic search. We present theoretical and practical results on new variants for 
exploring state-space with respect to memory limitations. 

We establish O(logn) minimum-space algorithms that omit both the open and 
the closed list to determine the shortest path between every two nodes and study 
the gap in between full memorization in a hash table and the information-theoretic 
lower bound. The proposed structure of suffix-lists elaborates on a concise binary 
representation of states by applying bit-state hashing techniques. Significantly 
more states can be stored while searching and inserting n items into suffix lists is 
still available in 0{n log n) time. Bit-state hashing leads to the new paradigm of 
partial iterative-deepening heuristic search, in which full exploration is sacrificed 
for a better detection of duplicates in large search depth. We give first promising 
results in the application area of communication protocols. 



1 Introduction 

Heuristic search in large problem spaces inherently calls for algorithms capable of run- 
ning under restricted memory. We present new data structures and algorithms that face 
the memory vs. duplication elimination problem that still arises even if the exploration 
is directed. The class of memory-restricted search algorithms has been developed under 
this aim. The framework imposes an absolute upper bound on the total memory the algo- 
rithm may use, regardless of the size of the problem space. If the number of nodes with 
distance value smaller than the optimal solution path length is larger than this memory 
bound, storing the entire list of visited nodes is no longer possible. 

Iterative deepening A*, IDA* for short [H3], has proven effective to successively 
search the problem graph with bounded DFS traversals according to an increasing thresh- 
old for the tentative values. IDA* consumes space linear in the solution length. It does not 
use additionally available memory and traverses all generating paths. Unfortunately, the 
number of paths in a graph might be exponentially larger than the number of nodes such 
that the design of informative consistent heuristics and duplicate elimination remains 
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Table 1. The IDA* Algorihm implemented with a Stack. 



IDA*(s) 

Push(S', s, h{s)); U -t— h{s) 
while (U 7 ^ oo) 

U ^U'\U' ^ oo 
while (S / 0) 

{u,f{u)) ^ Pop(S) 
if (goal{u)) return (m, f{u)) 
for all V in r{u) 

if (/(m) + w{u, v) — h(u) + h{v) > U) 

if (/(m) + w(u, v) — h(u) + h(v) < U') 
U' -t— f{u) + w{u, v) — h{u) + h{v) 

else 

Push(S', V, f(u) + wiu, ii) — h{u) + h{v)) 



essential. If all merits are distinct, IDA* expands a quadratic number of nodes in the 
worst case. Although iterative deepening is limited to small integral weights it performs 
well in practice. Tabled] depicts a possible implementation of IDA* in pseudo-code: S 
is a stack for backtracking, U is the current threshold, and U' the threshold for the next 
iteration. The value w{u, v) is the weight of the transitition {u, v), h{u) and f{u) is the 
heuristic estimate and combined merit for node u, respectively. 

Pattern data-bases 0 are a general tool to improve the estimate that can cope with 
complex subproblem interactions. A solution preserving relaxation of the search problem 
is traversed prior to the search and the goal distances of all abstract states are kept as 
lower bound estimats for the overall problem within a large hash table. However, the 
application of this pre-compilation technique is limited to suitable domain abstractions 
that yield better results than on-line computations as findings in protocol verification JSl, 
Al-planning 0, and selected single-agent problems iTiH indicate. Therefore, to lessen 
memory consumption according to a large number of states is still a problem. 

Transposition tables are used to store and improve the distances until the memory 
bound has been reached m- However, when the memory is exhausted, IDA*’s time 
consumption is often stinged by uncaught duplicates. 

Different node caching strategies have been applied: MREC lED switches from A* 
to IDA* if the memory limit is reached. In contrast, SMA* ca reassigns the space by 
dynamically deleting a previously expanded node, propagating up computed /-values 
to the parents in order to save re-computation as far as possible. However, the effect of 
node cashing is still limited. An adversary may request the nodes just deleted. 

The paper is aimed to close this gap and is structured as follows: The first section gives 
an 0(logn)-space algorithm to search for the shortest path in graphs with uniform or 
small weights, with n being the total number of nodes in the problem graph. Suffix lists are 
a data structure for maximizing the number of stored states according to a given memory 
limit. The achieved result is compared to ordinary hashing and a derived information- 
theoretic bound. Bit-state state compaction, sequential hashing and partial search can 
substitute the transposition table of IDA* with a bit-vector table. Thereby, it is possible 
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Table 2. Computing the BPS Level. 



Divide-And-Conquer-BFS(s) 
for i <— 1 ton 
for I <— 1 ton 



Path(a, b, 1) 
if ((a, 6) e E) 



return true 



else 



if {Path{s, i, 1)) 
print (s, i, 1) 
break 



for j •«— 1 to n 

if {Path{a,j, \l/2]) and Path(j, b, \ l/2\)) 



return true 



return false 



to detect more duplicates in the space while increasing the depth of the search. We give 
promising experimental results in validating an industrial communication protocol. 

2 Minimum Space Algorithms 

First of all, we might ask for the limit of space reduction. Given a graph with n nodes we 
are interested in algorithms that compute the BFS-level and shortest paths of all nodes 
and either consume as little working space as possible or perform faster if more space 
is available. In addition, we assume that the algorithms are not allowed to modify the 
input during the exection. 

The similar problems of node reachability (i.e., determine whether there any path 
between two nodes) and graph connectivity have been efficiently solved for the same 
restricted memory setting using random walk strategies [MU. However, we are not 
aware of any equivalent results for BFS and shortest paths. In the following we will devise 
an 0(log n) space algorithm for BFS and shortest paths with small integer weights. The 
principle is similar to the simulation of nondeterministic Turing machines coil. 

2.1 Divide-and-Conquer BFS 

To compute the breadth-first-level for each node, with very limited space, we may use a 
DAC strategy Path that reports if there is a path from a to 6 with I edges. If I equals 1 and 
there is an edge from a to 6 then the procedure returns true. Otherwise, for each node 
index j, 1 < y < n, we recursively determine Faf/!(a, y, andPath{j, b, [V2J)- If 

both exist the returned value is true, compare TableQ The recursion stack has to store at 
most 0(log n) frames each of which contains 0(1) integers. Hence the space complexity 
is 0(log n) . However, this has to be paid with a time complexity of 0(n^“''*°® ") due to the 
recurrence equation T(l) = 1 and T(l) = 2n ■ T{l/2) resulting in T(n) = (2n)^°®"’ = 

„l+logn 

time for one test. Varying b and iterating on I in the range of {1, . . . , n} gives 
the overall performance of steps. 

2.2 Divide-and-Conquer SSSP 

To extend this idea to the single-source shortest path problem (cf. Figure 01 with edge 
weights bounded by a constant C, we check the weights 
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Table 3. Searching the Shortest Paths. 



Path(a, 6, w) 



Divide-And-Conquer-SSSP(s) 



if {weight{a, b) = w) 

return true 



for i •«— 1 to n 

for w <— ItoC ■ n 



else 

for j <— 1 ton 

for s •«— max{l, \ w/2\ — [(7/2]} 



if (Path{s, i, w)) 
print (s, i, w) 
break 



to min{w — 1, \w/2\ + [(7/2]| 
if (Path(a, j, s) and Path(j, b,w — s)) 

return true 



return false 



[w/2j — [(7/2] for path 1, 
\_w/2\ — [(7/2 + 1] for path 1 



\w/2\ + [(7/2] for path 2, 
[z(;/2j + [(7/2] — 1 for path 2, 



[w/2j + [(7/2] for path 1, 



[tz;/2j — [(7/2] for path 2. 



If there is a path with total weight w then it can be decomposed into one of above 
partitions. The worst-case reduction on weights is Cn — ?• Cnj2 + (7/2 — ?> Cnj^, + 
3(7/4 — >^(7— )>(7— 1— )>(7 — 2— >^(7 — 3— )>...— >^1. Therefore, the recursion 
depth is bounded by log((7n) + C which results in a space requirement of (7(logn) 
integers. As in the BPS case this compares to exponential time. 

We do not claim practical applicability of these algorithms but want to make a start 
towards efficient shortest path algorithms for relatively little memory and unmodihable 
large data, for example on optical read-only storage. In particular, time-space trade-offs 
seem to require new techniques. 



Given m bits of memory, we want to maintain a dynamically evolving visited list closed 
under inserts and membership queries. The entries of closed are integers from {0, nj. Let 
r denote the maximal size of closed nodes that can be accommodated. As long asn < m 
a simple bit array with bit i denoting element i is sufficient. Using hashing with open 
addressing, r is limited to (7(n/ log n). In the following we describe a simple but very 
space efficient approach with small update and query times. Similar ideas appeared in 
Q but the data structure there is static and not theoretically analyzed. Another dynamic 
variant achieving asympotically equivalent storage bounds as our approach is sketched 
in P). However, constants are only given for two static examples. We provide constants 
for the dynamic version; comparing with the numbers of II , our dynamic version could 
host up to five times more elements of the same value range. However, one has to take 
into consideration that the data structure of dD provides constant access time whereas 
our structure incurs amortized logarithmic access time. 



3 Suffix Lists 
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closed nodes 

1011001 

0011011 

1011010 

0011101 

1011011 

0101000 

1011110 

0101001 

0101111 



in sorted order 



0011 


on 


0011 


101 


0101 


000 


0101 


001 


0101 


111 


1011 


001 


1011 


010 


1011 


oil 


1011 


110 



prefix-list suffix-list 



0000 


0 




1 


on 




0 




0 


101 




0 




1 


000 


0011 


1 


'' 


0 


001 




0 




0 


111 


0101 


1 




1 


001 




0 




0 


010 




0 




0 


on 




0 




0 


no 




0 










0 








1011 


1 










0 










0 










0 








nil 


0 









Fig. 1. Example for Suffix Lists with p = 4, and s = 3. 



3.1 Representation 

Let bin(u) be the binary representation of an element u < n from the set closed. We 
split bin{u) in p high bits and s = [log n] — p low bits. Furthermore, Ug+p-i, ■ ■ ■ ,Ug 
denotes the prefix of bin{u) and Ug-i, ■ ■ ■ ,uq stands for the suffix of bin{u). 

A suffix list data structure consists of a linear array P of size 2^ bits and of a two- 
dimensional array L of size r{m + 1) bits. The basic idea of suffix lists is to store a 
common prefix of several entries as a single bit in P, whereas the distinctive suffixes 
form a group within L. P is stored as a bit array. L can hold several groups with each 
group consisting of a multiple of s + 1 bits. The first bit of each s + 1-bit row in L 
serves as a group bit. The first s bit suffix entry of a group has group bit one, the other 
elements of the group have group bit zero. We place the elements of a group together in 
lexicographical order, see Figure [I] 

3.2 Searching 

First, we compute k = Us+i ■ 2* which gives us the search position in the prefix 

array P. Then we simply count the number of ones in P starting from position P[0] 
until we reach P[k]. Let z be this number. Finally we search through L until we have 
found the zth suffix of L with group bit one. If we have to perform a membership query 
we simply search in this group. Note that searching a single entry may require scanning 
large areas of main memory. 



3.3 Inserting 

To insert entry u we first search the corresponding group as described above. In case u 
opens a new group within L this involves setting group bits in P and L. The suffix of u 
is inserted in its group while maintaining the elements of the group sorted. Note that an 
insert may need to shift many rows in L in order to create space at the desired position. 
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The maximum number r of elements that can be stored in S bits is limited as follows: 
We need 2^ bits for P and s + 1 = [log n] — p + 1 bits for each entry of L. Hence, we 
choose p so that r is maximal subject to 

m-2P 

r < . 

[log n] - p + 1 

Forp = 6>(log?n — loglog(n/m)) the space requirement for both P and the suffixes in 
L is small enough to guarantee r = 0 ^ log^/m ) ) ■ 



3.4 Checkpoints 

We now show how to speed up the operations. When searching or inserting an element u 
we have to compute 2 in order to find the correct group in L. Instead of scanning 
potentially large parts of P and L for each single query we maintain checkpoints, one- 
counters, in order to store the number of ones seen so far. Checkpoints are to lie close 
enough to support rapid search but must not consume more than a small fraction of the 
main memory. For 2^ < r we have z < r for both arrays, so [log r] bits are sufficient 
for each one-counter. 

Keeping one-counters after every l/(ci • [logrj) entries limits the total space re- 
quirement. Binary search on the one-counters of P now reduces the scan-area to compute 
the correct value of 2 ; to ci • [log r\ bits. 

Searching in L is slightly more difficult because groups could extend over 2^* entries, 
thus potentially spanning several one-counters with equal values. Nevertheless, finding 
the beginning and the end of large groups is possible within the stated bounds. As we 
keep the elements within a group sorted, another binary search on the actual entries is 
sufficient to locate the position in L. 

3.5 Buffers 

We now turn to insertions where two problems remain: adding a new element to a group 
may need shifting large amount of data. Also, after each insert the checkpoints must 
be updated. A simple solution uses a second buffer data structure BU which is less 
space efficient but supports rapid inserts and look-ups. When the number of elements 
in BU exceeds a certain threshold, BU is merged with the old suffix lists to obtain 
a new up-to-date space efficient representation. Choosing an appropriate size of BU, 
amortized analysis shows improved computational bounds for inserts while achieving 
asymptotically the same order of phases for the graph search algorithm. 

Note that membership queries must be extended to BU as well. We implement BU 
as an array for hashing with open addressing. BU stores at most C 2 • r/ [log n] elements 
of size p-\- s = [log n] , for some small constant C 2 . As long as there is 10% space left 
in BU, we continue to insert elements into BU otherwise BU is sorted and the suffixes 
are moved from BU into the proper groups of L. The reason not to exploit the full hash 
table size is again to bound the expected search and insert time within BU to a constant 
number of tests. 
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Theorem 1. Searching and inserting n items into suffix lists under space restriction m 
can be done in 0{n ■ log^ n) bit operations. Assuming log n bits for a machine word, 
the total run time for n inserts and memberships is 0{n log n). 

Proof For a membership query we perform binary searches on numbers of [log r] bits 
or s bits, respectively. So, to search an element we need 0(log^ r + s^) = 0(log^ n) 
bit operations since r < n and s < log n. 

Each of the 0(r/logn) buffer entries consists of O(logn) bits, hence sorting the 
buffer can be done with 

( r ^ \ 

log n ■ • log = 0(r ■ log n) 

log n log n ) 

bit operations. Starting with the biggest occurring keys merging can be performed in 
0(1) memory scans, 0{m) operations. This also includes updating all one-counters. In 
spite of the additional data structures we still have 

r = e( ^ ). 

Vlog(n/m)/ 

Thus, the total bit complexity for n inserts and membership queries is given by 

0{ffibuffer-runs {ffisorting-ops + ffimerging-ops) + 

^elements ffibuffer-search-ops + 

^elements ffimembership-query-ops) = 

0(njr • log n ■ {r ■ log n + m) + n ■ log^ n + n ■ log^ n) = 

0(njr ■ logn • (r • logn + r • log(n/m)) + n ■ log^ n) = 

0{n ■ log^ n). 

Assuming a machine word length of log n in the RAM model, any modification or 
comparison of entries with 0(log n) bits appearing in our suffix lists can be done using 
0(1) machine operations. Hence the total complexity reduces to O (n • log n) operations. 

The constants can be improved using the following observation: in the case n = 
(1 + e) • TO, for a small e > 0 nearly half of the entries in P will always be zero, namely 
those which are lexicographically bigger than the suffix of n itself. Cutting the P array 
at this position leaves more room for L which in turn enables us to keep more elements. 

3.6 The Information Theoretic Bound 

We place an upper bound on the maximal size r* of the subset that can be stored. For the 
static case, we observe that [log (J!)] < to. However, if we consider the dynamic case, 
i.e. including insertions, we have to represent all former configurations. This results in 




< TO. 
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We aim choose r* maximal subject to this inequality. For r* < {n — 2) /3 we have 




The correctness follows from < 1/2 for i < {n — 2)/3. We are only 

interested in the logarithms, so we conclude 




Obviously in this restricted range it is sufficient to concentrate on the last binomial 
coefficient. The error in our estimate is at most one bit. The restriction on r* is compatible 
with all reasonable choices for n and m. Using 

log = iog "'<"-^- 

n r* 

= X! log/ - 

we can approximate the logarithm by two corresponding integrals. If we properly 
bias the integral limits we can be sure to compute a lower bound 

nn />r* + l 

> / log(x) dx — log(x) da;. 

Jn-r*-\-l J2 

Maximizing r* with respect to this equation yields an information theoretic upper 
bound. 

Table 0 compares suffix lists with hashing and open addressing. The constants for 
suffix lists are chosen so that 2 • ci + C 2 < 1/10 which means that if r elements can 
be treated, we set aside r/10 bits to speed-up internal computations. For hashing with 
open addressing we also leave 10% memory free to keep the internal computation time 
moderate. When using suffix lists instead of hashing, note that only the ratio between n 
and m is important. For the static data structure of Ql the following numbers are given: 
for ^ ~ 1.05 it can store a fraction of ^ ~ 4.37% of n. Our 

approach achieves 22.7% which constitues an improvement by a factor of more than 
five. For another example with n/mK, 3.2 our approach gains by a factor of about 1.8. 

Hence, suffix lists can close the phase gap in search algorithms between the upper 
bound and trivial approaches like hashing with open addressing. Already for n > 1.1 • m 
we reach two-optimality. 

4 Bit-State Hash-Tables 

Advanced to the treatment of data structures and algorithms we give a small introduction 
to the verification of distributed software systems and communication protocols; an 
apparent and practical relevant domain for state-space search. 
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Table 4. Fractions of n stored in Suffix Lists and Hashing with Open Addressing. 



n/m 


Upper 

Bound 


Suffix 

Lists 


Hashing 

n = 2^° n = 


1.05 


33.2 % 


22.7 % 


4.3 % 


2.9% 


1.10 


32.4 % 


21.2 % 


4.1 % 


2.8% 


1.25 


24.3 % 


17.7 % 


3.6% 


2.4% 


1.50 


17.4 % 


13.4 % 


3.0% 


2.0% 


2.00 


11.0% 


9.1 % 


2.3 % 


1.5 % 


3.00 


6.1 % 


5.3 % 


1.5 % 


1.0% 


4.00 


4.1 % 


3.7 % 


1.1 % 


0.7% 


8.00 


1.7 % 


1.5 % 


0.5 % 


0.4% 


16.00 


0.7% 


0.7% 


0.3 % 


0.2% 



4.1 State Space Search for Protocols Validation 

Reliable communication is probably the most important issue for accessing the Internet 
and for the design of distributed computer systems. Usually a layered structure like the 
ISO Reference Model is used to allow for different abstractions. In one layer (transport 
layer) we have the request for reliable communication while the next lower layers provide 
this quality of service facing a lossy channel (cf. Figure 0). 




Fig. 2. Communication over a Lossy Channel for Messaging in Layered Protocols. 



One example to cope with lossy channels is the alternating bit protocol. The message 
flow is visualized in Figure 0 To assert secure data transport from the sender to the re- 
ceiver we assume sequence numbers for messages. In the following we study algorithms 
and data structures to certify the correctness of a such a protocol. 

4.2 Supertrace 

The idea of bit-state hashing is adopted from Holzman’s protocol validator Spin O.that 
parses the expressive concurrent Promela protocol speciflcation language. It compresses 
the state description of several hundred bits down to only a few bits to build a hash table 
with up to entries and more. Combined with a depth-first search strategy this is in 
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Fig. 3. Flow of Control on a Lossy Channel with the Alternating Bit Protocol. 



fact the supertrace algorithm: A state s is represented by its hash address h{s). When 
generating a state the corresponding bit is set. Synonyms are regarded as duplicates 
resulting in pruning the search. The search algorithm is not complete, since not all 
synonyms are disambiguated. Moreover, through depth-first traversal, the length of a 
witness for an encountered error state is not minimal. 

4.3 Data Structures 

As an illustration and generalization of the bit-state hashing idea. Figure 0 depicts the 
range of possible hash structures: Usual hashing with chaining of synonyms, single-bit 
hashing, double-bit hashing and hash compact fzl. Let n be the number of reachable 
states and m be the maximal number of bits available. A coarse approximation for single 
bit-state hashing coverage with n < m is 1 — Pi with the average probability of collision 
Pi < - ~ — n/2m, since the i-th element collides with one of the i — 1 already 

inserted elements with a probability of at most (i—1) /m, 1 <i <n ITU . For multi-bit 
hashing and h (independent) hash-functions by assuming hn < m coverage is improved 
to 1 — Pfi with average probability of collision < ^ Y7=o ( ^ ■ m ) ^ ’ * elements 

occupy at most hi/m addresses, 0 < i < n — 1. For double bit-state hashing this 
simplihes to P 2 < Yl/Zo = ^(n — l)(2n — l)/3m^ < 4n^/3rn^. 

4.4 Sequential and Universal Hashing 

The drawback in incompleteness of partial search is compensated by re-invoking the 
algorithm with different hash functions to improve the coverage of the search tree. 
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Hash Table 



State s 



Bit-Address 




Hash Table 



State s 



A 

h(s) 

Bit-Address 




State s 



Hash Table 




Fig. 4. Ordinary Hashing, Single Bit-State Hashing, Double Bit-State Hashing, and Hash- 
Compact. 



Subsequently, this technique, called sequential hashing, examines various beams in the 
search tree (up to a certain threshold depth). In considerably large protocols supertrace 
with sequential hashing succeeds in finding bugs but still returns long witness trails. If in 
sequential hashing exploration with the first hash first function covers m/n of the search 
space, the probability that a state x is not generated in d independent runs is (1 — 
such that a; is reached with probability 1— {l — m/n)‘^. Eckerle and Lais (J| have shown 
that this ideal circumstances are not given in practice and refine the model for coverage 
prediction. 

Moreover, universal hash functions suit best for implementing sequential hashing. 
Let A, B be sets with \B\ — 2*", for some integer value w. The class of hash functions 
TL is universal, if for all x,y G A, we have 
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Universal hash functions lead to a good distribution of values on the average. If h 
is drawn randomly from TL and S is the set of keys to be inserted in the hash table, the 
expected cost of each search, insert and delete operation is bounded by (1 + |S'|/|i3|). 
We give an example of a universal hash function. Let p be prime, and p > |A| and 
hm,n{s) = {{m ■ s + n) mod p) mod \B\. Then the class TLi := {hm,n\'m, n S Zp] is 
universal. 

4.5 Validating Process 

For the validation of the design of the protocols, bug-hnding by simulation and testing has 
its drawbacks, since several subtle bugs in concurrent systems are difficult to establish. 
Given a formal specification of a desired protocol property model-checking is a push- 
button procedure to verify the correctness. Validation is performed by traversing the 
hnite-state machine representation of the protocol to hnd a bug. Therefore protocols are 
represented by state spaces, in which reachability analysis is performed to establish error 
states. 

Therefore, directed search for minimal counterexamples in the protocol space ac- 
cording to a given implementation corresponds to the search for an optimal solution 
with the goal as the failure state. From a model checking perspective Q the approach 
allows to implement various heuristics to direct the search into the direction of the 
failure. From an Al-perspective partial search, maybe assisted with sequential hashing, 
condenses duplicate information in various search and planning problem spaces. 

4.6 Heuristic Search Algorithm 

The apparent aspirant for state compaction is IDA* with transposition tables, since, 
in opposite to A*, it tracks the solution path on the stack, which allows to omit the 
predecessor link in the state description of the set of visited states. 

When substituting the transposition table H of already visited nodes in IDA* by 
bit-state, multi bit-state or hash compaction we establish the Partial IDA* algorithm 
as depicted in Table 0 Since neither the predecessor nor the /-value are present, in 
order to distinguish the current iteration from the previous ones, the bit-state table has 
to be re-initialized in each iteration of IDA*. Refreshing large bit- vector tables is fast in 
practice, but for shallow searches with a small number of expanded nodes this scheme 
can be improved by invoking ordinary IDA* with transposition table updates for smaller 
thresholds and by applying bit-vector exploration in large depths only. 

In practice the obtained counterexamples are minimal, since the coverage with bit- 
state duplicate elimination is very close to 100 % for moderately sized systems (n < m). 
Moreover, the technique of trail-directed search can effectivly improve non-optimal 
existing paths | 3 ]. 

The results for searching deadlocks in one large communication protocol are depicted 
in Table 0 where the number of expansions with respect to different optimal search 
algorithms for an increasing threshold is shown. For A* a snapshot is taken at each 
time the priority queue value increases, while in IDA* the number of expanded nodes 
according to each completed iteration is shown. Hence, the number of node expansion 
numbers in these two algorithms do not match exactly, but Indicate a common trend. 
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Table 5. Partial IDA* Algorithm. 



Partial IDA*(s) 

Push(S', s, h(s)); t/' •«— (7 t— h{s) 
while (U 7 ^ oo) 

U ^U'\U' ^oo 
Init(iT) 
while {S / 0) 

(«,/(«)) PopCS) 
if (goal{u)) return {u, f{u)) 
for all V in T'(m) 

if (Search(fT, v) 7 ^ 0) 

Insert (iT, v) 

if (/(u) -I- w{u, v) — h{u) + h{v) > U) 

if (/(m) -I- w{u, v) — h(u) -I- h(v) < U') 
U' -t— f{u) + w{u, v) — h{u) -\- h{v) 

else 

Push(S', V, fiu) + w{u, v) — h{u) + h{v)) 



The considered protocol instance is the industrial General Inter-ORB Protocol (GIOP, 
1 server and 3 clients) ca, which is a key component of the Common Object Request 
Broker Architecture (CORBA) specification. 

The witness for a seeded deadlock in depth 70 has to be established according to 
the heuristic that counts the number of non-active processes. The state vector generated 
by the validator tool SPIN is 544 Bytes large, such that the visited list (hash table or 
transposition table) is bounded to 2^® states corresponding to approx. 2^^ KByte or 128 
MByte. Therefore, we fix the size of the bit-state hash table accordingly at 2®° Bits. 

Algorithm A* exceeds its space limit in depth 61 and fails. IDA* utilizes a trans- 
position table which is exhausted at the same depth. As IDA* then searches the tree of 
generation paths it compensates space for time. But even when investing more than 24 
hours on our 248 MHz Sun Ultra Workstation and when utilizing the table constructed so 
far, ordinary IDA* was not able to complete search depth 61. On the other hand. Partial 
A* finishes all searches up to depth 70 with either single- and double bit-state hashing 
within a total of one hour. 

Since the algorithms are not complete, we validated optimality with A* with our 
maximum of 1.5 GByte main memory. Note that the difference in the number of node 
expansions in single and double bit-state hashing is very small (less than a hundred) and 
only occurs in large search depths (iteration 58 onwards). As Partial IDA* with double 
bit-state hashing expands exactly the same number of states as IDA* with a transposition 
table, we actually observe no loss of information in the example. 

5 Conclusion 

At the limit of main memory eliminating duplicates and weight diversity can soon result 
in thrashing both resources time and space, such that powerful data structures for caching. 
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Table 6. Number of Expanded nodes of Search Algorithms in the GIOP Protocol. 



depth 


A* 

(hash table) 


IDA* 

(transposition table) 


Partial IDA* 
(single bit-state) 


Partial IDA* 
(double bit-state) 


40 


6,646 


6,333 


6,333 


6,333 


41 


9,306 


8,184 


8,184 


8,184 


42 


10,955 


10,575 


10,575 


10,575 


43 


13,666 


13,290 


13,290 


13,290 


44 


17,761 


16,500 


16,500 


16,500 


45 


20,130 


19,860 


19,860 


19,860 


46 


25,426 


23,646 


23,646 


23,646 


47 


27,714 


27,654 


27,654 


27,654 


48 


33,799 


32,040 


32,040 


32,040 


49 


37,095 


37,011 


37,011 


37,011 


50 


46,105 


42,849 


42,849 


42,849 


51 


51,113 


49,872 


49,872 


49,872 


52 


61,710 


58,545 


58,545 


58,545 


53 


73,195 


69,162 


69,162 


69,162 


54 


85,245 


81,993 


81,993 


81,993 


55 


96,995 


96,543 


96,543 


96,543 


56 


113,950 


112,296 


112,296 


112,296 


57 


115,460 


129,138 


129,138 


129,138 


58 


147,042 


146,625 


146,623 


146,625 


59 


150,344 


164,982 


164,978 


164,982 


60 


184,872 


184,383 


184,376 


184,383 


61 


187,411 


206,145 


206,135 


206,145 


62 


- 


> 97,157,721 


229,611 


229,626 


63 


- 


- 


255,386 


255,411 


64 


- 


- 


282,416 


282,444 


65 


- 


- 


311,306 


311,340 


66 


- 


- 


341,522 


341,562 


67 


- 


- 


373,374 


373,422 


68 


- 


- 


407,249 


407,310 


69 


- 


- 


442,863 


442,941 


70 


- 


- 


67 


67 



partial search and compressed dictionaries are required. Therefore, regarding the limits 
and possibilities of A*, we have suggested different contributions to memory-restricted 
search. Partial search supports bookkeeping in tremendously large hash tables to avoid 
duplicates in the search, while suffix lists push the envelope for increasing the number 
of nodes to be stored without loss of information. 

The treatment of Partial IDA* search elaborates on precursoring findings in @, 
where a rudimentory bit vector and single-bit hashing function has been chosen for 
implementation. For the experiments we chose a non-trivial protocol example but 
recent progress shows that the algorithm has also reduced the search efforts for optimally 
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solving Atomix, a PSPACE-complete AI single-agent search problem Id. Omitting the 
visited list and exploring the space in a Divide-and-Conquer fashion has been proposed 
in C3, and the algorithms we consider study the effect of removing the horizon-list 
as well. Another model checking approach for state compression as to answer to the 
representation problem of large sets of states are binary decision diagrams (BDDs) that 
are able to encode large sets of states without necessarily encountering exponential 
growth. However, hybrid methods of explicite and symbolic search methods are still to 
be developed. 
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Abstract. Debugging, validation, and maintenance of configurator knowledge 
bases are important tasks for the successful deployment of product 
configuration systems. Consistency-based diagnosis has shown to be a 
promising approach for detecting faulty parts in the knowledge bases and 
explaining unexpected behavior of the configurator, whereby (partial) 
configurations are used as test cases. In this paper we show how hierarchical 
diagnosis can be employed to cope with the complexity of debugging large 
configurator knowledge bases. A framework for hierarchical diagnosis on 
different levels of abstraction is presented as well as an algorithm for the 
calculation of diagnoses on those levels. The presented approach aims at the 
reuse of existing special purpose configuration systems. We show that the 
exploitation of hierarchies in such problem domains leads to a significant 
efficiency enhancement thus broadening the applicability of consistency-based 
diagnosis. 



1 Introduction 

Knowledge-based configuration is an important application field of A1 technology 
due to increasing demand for configurable and mass-customized products. The 
increased complexity and high change rates of the products (and the corresponding 
knowledge bases) made adequate debugging and testing support a prerequisite for 
successful deployment of such tools. Consistency-based diagnosis techniques have 
shown to be applicable not only for diagnosing electronic circuits or other hardware 
devices, but also for debugging of software systems such as logic programs Q, 
repairing inconsistencies in databases [ITUl or VHDL programs pi. 

In Ipl it was shown how model-basea diagnosis (MED) can mso be employed for 
error detection within knowledge bases for configuration systems. A framework is 
described where "positive" and "negative" examples can be provided, where (partial 
or complete) "positive" examples should be accepted or completed by the 
configurator and "negative" ones should be rejected. The behavior of the configurator 
is unexpected, if a positive example cannot be completed or causes a contradiction or 
an intended negative example is falsely accepted. For localizing possible 
explanations, the problem is mapped to a diagnosis problem, where the sentences 
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(constraints) of the knowledge base play the part of components from the MBD 
terminology. 

The success of AI techniques in the configuration area is based on highly 
specialized configurators | |1 jp ) able to deal with large configuration problems. In 
addition such systems use a restricted first-order logical language. Our goal is to 
extend such specialized systems with diagnosis capabilities for debugging. The work 
of ITTFI shows how such an integration of special consistency-checking systems into a 
genem diagnosis problem solver can be achieved. However, the efficiency of the 
diagnosis computation depends heavily on the minimality of the conflict sets. 
Typically, specialized configurators offer no generation of conflict sets or provide 
non-minimal conflicts including many additional elements. One reason for this is that 
in configuration we have to deal with general clauses and many dependencies 
between these constraints. Therefore, dependency tracing on top of constraint 
propagation will not reduce the conflicts significantly in practice. 

In this paper we show that hierarchical abstraction (ni,li^,|i3|l,|i /|) significantly 
enhances the efficiency of diagnosing configuration feiowTMgeDases. Typical 
hierarchical structures in configurator knowledge bases can be employed for 
diagnosis at different levels of granularity. The paper is organized as follows. After 
giving a motivating example, we shortly review the framework of consistency-based 
diagnosis of configurator knowledge bases. We present the extension of this approach 
with structural abstractions for diagnosis at different levels. After the description of an 
algorithm and first results from a prototypical implementation, we discuss related 
work and present our conclusions. 



2 Motivating Example 

For demonstration purposes we use a small fragment of a configuration problem. 
After inserting a typical failure, we show how consistency-based diagnosis can help to 
detect this error. We use first order sentences to ensure clear representation with 
precise semantics. In the example, we have a frame, where cards of different types 
(CPUs and switching modules) can be inserted on existing named slots. The 
knowledge-base consists of the following definitions, where types describes the 
available component types and ports describes the predefined connection points for 
the components [|^ : 

types = {frame, cpu-1, cpu-2, sm-1, sm-2j; 
ports(frame) = (cpl,cp2,smpl,smp2j. 
ports(cpu-l) = {framepj. ports(cpu-2) = jframepj. 
ports(sm-l) = (framepj. ports(sm-2) = (framepj. 

We use the following predicates for describing configuration knowledge: type(c,t) 
associates a component identifier with one of the predefined types, conn(cl,pl,c2,p2) 
describes that component instance cl is connected on port pi with another component 
c2 on p2. Attribute valuations of components are described by a predicate val/3 which 
is omitted here (see ^ for an example). In addition, definitions are contained 
describing which components can be connected via which ports in general and other 
domain-independent constraints, like that one port can be connected to exactly one 
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other port and connections are symmetric. In addition, the following constraints for 
the individual component types have to hold: 

"CtFj: If there is a cpu-1 on cp2, there must be a sm-1 on one of the switching 
module ports. ", more formally 

VC,F : type(F,frame) a type(C, cpu-1) a conn(F,cp2,C,framep) 

3(S,P): type(S,sm-l) a conn(F,P,S,framep) a P e {smpl, smp2}. 

For sake of brevity, we only describe the other constraints without formal 
representation. 

"CtSj: If there is a switching-module sm-2 on smpl there must be also one sm-2 on 
smp2. " 

"CtCj: If there is a CPU of any type connected to any CPU port, at least one 
switching module of type sm-1 or sm-2 must be connected to smpl or smp2. " 

"CtC 2 -' A CPU of type cpu-2 on port cpl requires switching modules of type sm-2 
on both ports smpl, smp2. " 

"CtCs-'A CPU of type cpu-1 on cp2 requires a CPU of the type cpu-2 on cpl. " 

Let us assume, that CtF] is faulty and too restrictive, because also switching 
modules of type sm-2 should be allowed. This situation came about because sm-2 was 
a type newly introduced to the knowledge base and CtF] was not maintained 
correctly. The user provides a positive example with one cpu-1 and a switching 
module sm-2: 

e'^ = (3 F,S,C: type(F,frame) a type(S,sm-2) a 
type) C, cpu-1) A conn(F,cp-2,C,framep) a 
conn(F, smp-1, S,framep)}. 

Note, that the partial example cannot be completed to a correct configuration (See 
[PI for the usage of negative examples). Following the consistency-based approach 
from [Q the minimal conflicts 0 (sets of constraints causing a contradiction with 
the example) 

{CtF], CtS]} and (CtF], CtCj, CtC 2 j 
induce the minimal diagnoses for the unexpected behavior: 

{ab(CtF,).j (ab(CtSi). abiCtCs).} {ab(CtS,). abiCtCi).} 

In other words, if these sets of constraints (diagnoses) are considered abnormal and 
are canceled, the partial configuration can be completed by the configurator. 

Note, that CtC, is not contained in any minimal conflict set and that the assumption 
of having minimal conflict sets available for the HS-DAG (|nTl,|TP|) generation is 
very strong for the configuration domain for different reasonsTrirsL there are many 
interdependencies between the constraints, so a dependency analysis or tracing will 
not suffice. Second, for the configuration domain, tools (e.g., [1/^ ]) with specialized 
inference mechanisms {Generative Constraint Satisfaction [j^) and limited 
explanation facilities (and costly conflict minimization) are employed for effective 
calculation of configurations. In addition, the constraint language must be expressive 
enough for the domain, i.e., the language’s constructs exceed the expressiveness of 
//or«-clauses. Consequently, the conflicts returned by the specialized theorem prover 
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are large and non-minimal, leading to high search complexity during the HS-DAG 
generation, because overlaps in the conflict sets minimize benefits from techniques 
like conflict reuse. One could argue that we should minimize the conflicts before 
using them during HS-DAG generation. However, note that the computation of 
additional minimal conflicts is costly, because conflicts with many redundant 
elements may contain several minimal conflicts, such that minimization requires a 
number of consistency checks, which is not linear in the number of elements of the 
non-minimal conflict. 

Following a hierarchical approach, we will try to analyze the system on a coarse 
level with smaller complexity, i.e., we do not diagnose individual constraints but 
whole groups of related constraints. If we group the constraints (indicated by the 
names in the example) and assume all contained constraints to be faulty if such a 
named group is "abnormal", the minimal diagnoses will be 



A faulty constraint is simply not considered for consistency checking, i.e., we do 
not consider fault models. 

Having only three diagnosable components at the abstract level (i.e., the constraint 
groups /rame, sm, and cpu), this result can be calculated very fast. Then, the user can 
decide to use the result as a pointer to a faulty group or refine the diagnoses to the 
next level. When refining one diagnosis to the next level, the contained faulty groups 
are replaced by their elements; groups that were not assumed to be faulty are still 
treated as one component, which reduces the number of diagnosable components. 

If we extend the example to have five component types (groups), each containing ten 
constraints (diagnosable components n=50) and the diagnoses as before, the number 
of theoretically possible combinations for double-faults is 



This number is not reduced significantly when only large non-minimal conflicts are 
available. Using the hierarchical approach, for the top-level diagnosis there are only 
ten possible two-element combinations for five groups. Refining the first abstract 
diagnosis with replacement of the faulty group /rame leads to n=10+4 diagnosable 
components and n=20+3 for the second diagnosis. So, the theoretical upper limit for 
hierarchical approach for double faults without regarding conflicts is 



3 Diagnosing Configurator Knowledge Bases 

In our general framework, a configurator knowledge base consists of a set of logical 
sentences DD describing available component types, their attributes and connection 
points as well as constraints on legal product constellations ||14|. Configuration 
problems are solved according to specific user requirements A configuration 
result can be described by means of a set of ground literals containing information on 
component instances, attribute values and connections. The set of possible literals is 
contained in a set CONL. 



{ab( frame).} (ab(sm). ab(cpu).} 
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Definition: (Configuration problem): A configuration problem is described by a 
triple (DD,SRS,CONL), where DD and SRS are sets of logical sentences and CONL is 
a set of predicate symbols. 

DD represents the domain description, SRS the user requirements for a configuration 
problem instance. A configuration CONF is described by a set of ground literals 
whose predicate symbols are in CONL. o 

Definition (Consistent configuration): Given a configuration problem (DD,SRS, 
CONL), a configuration CONF is consistent iffDD u SRS u CONF is satisfiable. o 
To ensure the completeness of a configuration, additional formulae for each symbol in 
CONL have to be introduced to CONF, e.g., type(X,Y) => ( type(X,Y) e CONF). 

We denote the configuration CONF extended by these axioms with CONF . (For a 
detailed exposition, see 

Definition (Valid and irreducible configuration): Let (DD, SRS, CONL) be a 
configuration problem. A configuration CONF is valid iff DDu SRS U CONF is sa- 
tisfiable. CONF is irreducible if there exists no other valid configuration COND“ 
such that CONF'"‘’ czCONF. □ 

Definition (CKB-Diagnosis Problem): A CKB (Configuration Knowledge Base) 
Diagnosis Problem is a triple (DD,Ef,E~) where DD is a configuration knowledge 
base, Ff is a set of positive and E~ a set of negative examples given as sets of logical 
sentences. IVe assume each example on its own to be consistent. □ 

Positive examples are (partial) configurations, which should be accepted by the 
configurator, whereas negative examples should be rejected. Given these example sets 
and the domain description cause an inconsistency, a diagnosis corresponds to the 
removal of possibly faulty sentences restoring the consistency. In addition, if a 
negative example is consistent with the knowledge base, we have to find an extension 
to DD which restores inconsistency for all such negative examples. 

Definition (CKB-Diagnosis): A CKB-Diagnosis for a CKB-Diagnosis Problem (DD, 
E^,E~) is a set S c" DD such that there exists an extension EX, where EX is a set of 
logical sentences, such that 

DD - S uEX ue* consistent \/ e* e Ef 
DD - S uEX ue inconsistent e~ e E~ . □ 

From here on we refer to the conjuction of the negated negative examples as NE, i.e., 
NE = Af,f(^e-). 

Proposition: Given a CKB-Diagnosis Problem (DD, E'^, E~) , a diagnosis S exists iff 
\/e E r .• e* U NE is consistent. 

Proof, see |^. 

Corollary: S is a diagnosis iff 

Ve'^ e Ef : DD —Su e* U NE is consistent. □ 

The following remark relates configuration and diagnosis for configurator knowledge 
bases [Q. 

Remark: Let be partitioned in two disjoint sets and e^^^^ where is a set 
of positive ground literals whose predicate symbols are in the set of CONL and e^^^^ 
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represents system requirements (if some are specified in conjunction with the positive 
example). 

S is a diagnosis ( DD, E~) iff V s : e* is a consistent configuration for 

(NEuDD-S, , CONE). 

Note that if the completeness axioms have been added to then is a valid 
configuration for (NEuDD—S,e^^g^, CONE). 



4 Hierarchies in the Knowledge Base 

We will show how hierarchies in the knowledge base can be used for the calculation 
of diagnoses on the different levels of abstraction. Therefore we assume that the 
individual constraints from the knowledge base are arranged into named groups which 
can be again grouped, such that the structure forms a tree. This hierarchical structure 
T can be expressed using a function sons(n), which returns the direct successors of a 
node n in the tree, i.e., the elements of a named group n (and 0 if n is a leaf node). 
We assume a group root to exist in the tree representing the root of the tree. The leaf 
nodes of the tree are the individual sentences from DD. A function leaves(n) returns 
all leaf nodes for a given node n which are under n (and n itself if it is already a leaf 
node). Finally, all diagnosable constraints from DD have to be contained in the tree. 
Note, that the idea for the following framework is that we consider all constraints of a 
group to be potentially faulty, if at least one constraint of the group is faulty. 

Definition (Hierarchy tree): A hierarchy tree T for a configuration knowledge base 
DD is a tree, where 

the leaf nodes are named elements from DD, 
a node "root" represents the root element of the tree, 

inner nodes represent named constraint groups from the knowledge base, and 
the names all leaf nodes and inner nodes appear exactly once in the tree. □ 

For hierarchical diagnosis we extend our notion of CKB-Diagnosis in a way that 
also constraint group names can appear in the diagnosis. We define a function 
successors(n) to be returning the set of all direct and indirect successors of a node n in 
the tree (and 0, if « is a leaf node). The function allLeaves(N) defined on a set of 
nodes returns the union of leaves(n) applied to every ne N. 

Definition (Abstract CKB-Diagnosis): An Abstract CKB-Diagnosis for a 
configuration problem (DD,Ef,E~) and a hierarchy tree T is a set S of nodes of T, 
such that there exists an extension EX, where EX is a set of logical sentences, such 
that: 

DD— allLeaves(S) uEX ue* consistent V e* e E*, 

DD— allLeaves(S) uEX ue~ inconsistent V e~ e E~ .1^ 

Definition (Minimal Abstract CKB-Diagnosis): An Abstract CKB-Diagnosis S for 
(DD,E^,E~) and T is said to be minimal, if no subset S’ cz S is an Abstract CKB- 
Diagnosis. □ 
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In order to ensure that by using this form of abstraction for different levels no 
diagnostic information is lost, we have to show that every abstract level diagnosis has 
a corresponding diagnosis at a more detailed level. 

Proposition (Soundness of diagnosis): Let ABSTR be an Abstract CKB-Diagnosis 
for a configuration problem (DD,E^ ,E~) and a hierarchy tree T then there exists a 
CKB-Diagnosis DIAG such that DIAG is a subset of allLeaves( ABSTR). 

Proposition (Completeness of diagnosis): Let DIAG be a CKB-Diagnosis for a 
configuration problem (DD,E^ ,E~) and a hierarchy tree T then there exists an Abstract 
CKB-Diagnosis ABSTR such that DIAG is a subset of allLeaves(ABSTR). 

Proof: (see |^). 



5 Computing Diagnoses at Different Levels 



Given the above definitions, we can extend the standard hitting-set algorithm for 
model-based diagnosis to calculate (minimal) diagnoses at the different levels. In the 
standard algorithm (it l|,|iaj), conflict sets are used for focusing purposes. For the 
domain of diagnosis ot knowledge bases [^, a conflict set is defined as follows: 

Definition (Conflict Set): A conflict set CS for (DD, E'^,E~) is a set of elements from 
DD such that 

3 e"" e E'^ : CS Ue'^ U NE is inconsistent. □ 

In order to support calculation of minimal diagnosis at different levels of abstraction, 
we extend the definition, such that also constraint groups can appear in a conflict set. 

Definition (Abstract Conflict Set): An abstract conflict set for (DD,E^,E ) and a 
hierarchy tree T is a set ACS of elements from T such that 
3 e"" e Ef: allLeaves(ACS) LJe'*' uNE is inconsistent.^^ 

For the computation of minimal diagnoses for configurator knowledge bases, the 
HS-DAG algorithm from ( [11] , | |lb[ ) is adapted as follows: a node n in the DAG is 
labeled by a conflict set ACS(n)-, edges leading away are labeled by elements s e 
ACS(n). The set of edge labels on the path from the root to a node n is referred to as 
H(n). In addition, for each node n a set CE(n) of consistent positive examples is 
stored, knowing that once an example is already consistent it will not become 
inconsistent after further removal of constraints. Since a node can have multiple direct 
predecessors [11] - furred to as preds(n) - we combine the sets CE from all direct 
predecessors for sucna node. 

According to the idea of iteratively substantiating abstract diagnoses following the 
hierarchical structure of the problem, we will initially compute a set of high-level 
diagnoses, which can then be refined to a more detailed level. Consequently, the 
diagnostic algorithm has an additional input parameter (context) besides the problem 
description and the examples, i.e., an abstract diagnosis that was already computed on 
a higher abstraction level. For the calculation of diagnoses on the next level of detail, 
the constraint groups from the higher-level diagnosis are replaced by their successor 
nodes according to the hierarchy. Accordingly, given an abstract diagnosis AD as 
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context, the diagnosable components (in terms of model-based diagnosis typically 
denoted as COMPS) for the refined diagnoses are given as follows: 

• If AD = 0, only elements from sons(root) can be contained in the diagnoses. 

• If AD # 0, we have to take a special set of nodes into account for the next 
refinement step: a) the leaf nodes from AD, and b) for each constraint group in 
AD, we have to compute the path of that element to the root of the hierarchy tree. 
Given the set of nodes that are contained in one of these paths, we have to 
compute the union of all direct successors of these nodes. 

Note, that we have to take these direct successors along the abstraction hierarchy 
into account for the next-level diagnosis, since additional constraint groups leading to 
minimal diagnoses can appear in the detail-level diagnoses, which were hidden by the 
minimality criterion at some abstract level. These special cases of hidden diagnoses 
are explained in more detail in [6] . H 



Algorithm 1: Diagnosis in abstraction context (schema). 

In: {DD,EC,E~), T, an Abstract Diagnosis AD 
Out: a set of refined diagnoses RD 

(1) Use the hitting set algorithm to generate a pruned HS-DAG D for the collection F 
of abstract conflict sets for ((DD,EP,E~), T, AD). Compute the DAG in breadth- 
first manner in order to generate diagnoses in order of their cardinality. 

(a) Every theorem prover call TP(DD — H(n), PP - CE(preds(n)), E~, T, AD) at a 
node n tests whether there exists an e* e E* such that there is an inconsistency. 
In this case an (abstract) conflict set is returned, otherwise it returns ok. 

(b) Set CE(n) to be the set of examples found to be consistent in the call to TP 
union the already consistent examples at the direct precedessors of n. 

(2) Return (H(n) I n is a node of D labeled by ok}. 



6 Computing All Minimal Diagnoses 

We propose an iterative approach starting with a high-level diagnosis that can be 
computed efficiently. The user can decide to stop at this level and focus on some 
group(s) of constraints or can refine these results to a more concise level. In the 
following, an algorithm is presented where a tree with nodes labeled with sets of 
diagnoses is generated, where at each successor node one of the diagnoses of the 
parent is refined. First, an initial set of top-level diagnoses (in context root of 
hierarchy tree T) is generated. Then the tree is generated in breadth-first manner, 
where for each diagnosis of the parent still containing a constraint group, a child node 
is generated and diagnosis is performed in the context of that diagnosis. Note that the 
node is only refined if the considered diagnosis is not already somewhere else in the 
tree. The algorithm ends, if no more nodes can be refined. Furthermore, if we are only 
interested in leading diagnoses, the search can be limited, e.g., to a given cardinality. 
The usage of the standard diagnosis algorithm guarantees that the computed diagnoses 
are correct and minimal. Furthermore, the result of every refinement step 
characterizes the candidate space. These candidate spaces include all minimal 
diagnoses. It follows, that no minimal diagnosis is excluded during refinement. 
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Algorithm 2: Iterative refinement of diagnoses( sketch): 
rootnodejdiagnoses = diagnose(DD,E^,E~, T, 0) 
set E* = E* — I e* s E*\ e* consistent with DDf 
. label refine 

refinable = set of diagnoses from current leaf nodes 
containing constraint groups, 
if refinable = 0 goto :end; endif 
forall d e refinable 

calculate diagnosis d’ = diagnose(DD,E^ ,E~ , T,d) 
if d’ not already in tree 
create child node for d labeled with d’ 

endif 
endfor 
goto : refine 
. label end 



diagnose(DD,E'^,E" , T, 0 ) 



1 




Fig. 1. Tree of diagnoses for example problem 

Fig. 1 depicts the outcome of the application of Algorithm 2 to the example problem 
from Section 2. Fig. 2 shows the hierarchy tree T for the example. 




Fig. 2. Hierarchy tree T for example problem 

Diagnosis at the top level results in two diagnoses (ab(sm). ab(cpu).j and 
{ab(frame).}\ these diagnoses are contained in the root node of the tree in Fig. 2. 
Next, these diagnoses are refined in breadth-first order, i.e., {ab(sm). ab(cpu).j is 
expanded resulting in three diagnoses, namely jab(CtSfi ab(CtCJ.j, { ab(CtSJ. 
ab(CtCJ.j, and {ab(frame)}. The last one is not added to the refinement tree because 
it is already contained in the root node of the tree. After that, {ab(frame).j is 
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expanded, resulting in lab(CtFJ.} and jab(cpu). ab(sm).}, whereby the latter is again 
not included in the tree. Next, the algorithm proceeds by trying to refine the diagnoses 
at the next level; however, in this case, this second level in the tree does not contain 
any diagnoses that contain refinable components, and the algorithm ends. 



7 Implementation 

In order to test the applicability of our approaches, we implemented a prototype using 
the industrial- strength software library ILOG Configurator |T7|. Using this package 
of C-H- libraries, a configuration problem is formulated iuTerms of a Generative 
Constraint Satisfaction Problem (GCSP) 171 ]. This enhancement of the basic CSP 
mechanism allows the number of variables ot the problem to be dynamically changed, 
and the number of employed components may not be known beforehand. The domain 
description with the additional constraints and the examples are declared to the 
configurator using calls to the library. In the context of that CSP, a conflict set is a set 
of constraints from the knowledge base, which, if canceled, makes the configuration 
problem satisfiable. Using this library, the search for an arbitrary solution for a 
configuration problem can be done very efficiently. However, no adequate 
explanation mechanisms for the calculation of (minimal) conflict sets are provided. 

We implemented the diagnostic algorithm both for the flat approach from and 
the extended hierarchical algorithm. Note, that when using this library, a basic two- 
level hierarchy, i.e., assignment of constraints to component types is already given 
with the problem formulation and does not cause additional modeling efforts. The 
effective computation time for the diagnosis task depends on several factors, e.g., 
number of constraint types, cardinality of the diagnoses or the time to test one 
individual example for satisfiability. Diagnosis of the simple example problem can be 
done nearly instantaneously with both algorithms; the identification of two triple 
faults in a setting with about twenty types of constraints and about hundred constraint 
instances is done in a few seconds on a standard PC running Windows NT with both 
algorithms, whereby our unoptimized prototype does not calculate minimal conflict 
sets nor utilizes domain dependent heuristics. However, for larger knowledge bases 
(containing about hundred types of constraints and component types and hundreds of 
constraint instances), the usage of the hierarchical approach with refinement of the 
diagnoses leverages the problem of computational complexity. When considering our 
simple example from Section 2, components will only be considered as one single 
diagnosable element and will never be expanded to a more detailed level if they are 
not in any abstract diagnosis. Therefore, even larger and complex knowledge bases 
remain diagnosable within an acceptable computation time of a few seconds, because 
additional constraints within the correct parts of the knowledge base only influence 
the costs of the consistency checks but not those of the diagnostic process. In 
addition, we conducted preliminary experiments with real-world examples from the 
domain of private telecommunication systems. 

Complexity issues. For a simple analysis of the reduction of the computational 
complexity when using a hierarchical approach, let us consider the search space for 
diagnosis candidates for both approaches. Given a set of n diagnosable components, 
i.e., configuration constraints, where we want to find diagnoses of cardinality k, the 
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computational complexity is 0(t/). If we are given a two-level hierarchy where the n 
diagnosable components are distributed over g groups, the number of groups 
determines the computational complexity O(g^) for the first level. If we want to refine 
one of the diagnoses, the constraint groups are replaced with the contained elements 
from the next level and the average number of elements for two levels will be n/g. 
Given a number s, s < k < g, describing the number of constraint groups in the 
abstract diagnosis, the number of diagnosable components for the next level is (n/g)*s 
+ (g- s). The number of remaining constraint groups, which were not in the abstract 
diagnosis and are therefore not refined, is (g - s). This leads to the complexity of 
0((n/g)*s + {g-s)t). 

The possible achievable benefits from the hierarchical abstraction depend on 
several factors: First, the number of needed refinement steps depends on the number 
of existing diagnoses (the upper bound) and the distribution of elements from the 
detailed diagnoses among constraint groups. In the best case, all detail-level diagnoses 
are included in one (or a few) abstract diagnosis, whereas in the worst case all 
detailed diagnoses correspond to different abstract diagnoses. However, it can be 
reasonably assumed that only small fractions of the knowledge base are faulty after 
maintenance. Another factor is how the constraints are grouped, i.e., how many 
constraints for an individual group exist. In the worst case, each group contains one 
element leading to additional overhead when using the hierarchies. In good cases, 
only groups of small size are contained in the abstract diagnoses. 

Finally, the assumption that only large, non-minimal conflicts are available 
following the argumentation of Section 2 (dependencies, expressiveness of config- 
uration language, and specialized inference mechanisms) is an important factor. In 
cases where minimal conflicts can be easily computed, the hierarchical approach will 
lead to additional overhead if diagnoses at the detailed levels are needed. Conversely, 
the more the conflicts contain irrelevant elements, the better the hierarchical approach 
reduces the complexity. 



8 Related Work 

Model-based diagnosis techniques were initially developed for the identification of 
faults in physical devices, e.g., electronic circuits. Later, these techniques were 
adopted for diagnosis and debugging of software, e.g., logic programs [4nrelational 
database consistency constraints [10]^lrdware designs specified in VHDL [9], w 
overconstramed Constraint SatisfactionTTroblems [2].|C]ur work extends the work of 
[5] riy exploiting hierarchies for consistency-basea diagnosis of configuration 
knoMedge bases. The usage of hierarchies for the diagnosis task has been discussed 
in various application areas of model-based diagnosis (e.g., [|5],[jr71,[p31,|n71). Our 
approach mostly corresponds to what is called structural abstmcnon (vSr oemvioral 
abstraction) and aims at a more efficient diagnosis process. One of the important 
problems is to have the information on the hierarchy available at each abstraction 
level (causing additional modeling effort). For the case of debugging of configuration 
knowledge bases, however, the hierarchical abstraction has a good correspondence to 
the configurable artifact. Changes to the product catalog are usually applied to sets of 
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modules (configuration components) leading to a small set of effectively affected 
components. 

In 0, it was shown that when modeling a system at different levels of abstraction 
(independently) for general diagnosis problems there may be situations where 
diagnoses at a detailed level do not have a correspondence to a diagnosis on a more 
abstract level such that diagnostic information may be lost. This phenomenon cannot 
appear in our approach, because at each level, the system’s "behavior" (consistency 
checks) is always analyzed on the most detailed level. 

El describes hierarchical diagnosis based on value propagation and with XDE an 
extension of the ATMS approach. Our approach is similar in the way structural 
decomposition is applied. However, our goal was to integrate configuration engines 
(e.g., based on generative constraint satisfaction) and diagnosis. The approach of 
offers an appealing way for this integration, which was extend by our work in order to 
employ hierarchies. [1151 also uses hierarchies for improving the efficiency of 
diagnosis but applies a'mfferent notion of diagnosis by defining a diagnosis as a 
logical consequence of a theory. 

Different approaches to diagnosis which avoid the computation of conflict sets 
were proposed by H] and [1T^ ]. They improve the underlying theorem proving 
algorithms such that magnosesTan be computed efficiently. Note, that our goal was to 
reuse specialized problem solvers, which are optimized to solve complex 
configuration problems and not to provide conflict sets of explanations. The 
incorporation of diagnosis techniques from [Oand [1 |»J p ithout degrading the perfor- 
mance of the configurators remains an interesmig open issue. 



9 Conclusions 

The demand for (Al-based) product configuration technology is steadily increasing. 
For validation and maintenance tasks only limited support can be found in nowadays 
systems. For these tasks it was shown in how techniques from model-based 
diagnosis can support the knowledge engineer m validating the knowledge base. 

Currently, for the configuration of large and complex products, special problem 
solving mechanisms must be applied. In addition, the domain requires general clauses 
for expressing configuration constraints. Due to these facts, only limited explanation 
of conflicts is provided. We showed how the exploitation of hierarchical structures 
can significantly improve the efficiency of diagnosing configuration knowledge 
bases. This was achieved by employing specialized configuration systems for 
consistency checking and extending the approach from 0 in order to cope with 
hierarchies. We have presented a sound and complete algorithm relying on iterative 
refinement of diagnoses and validated our approach in a prototype implementation. 
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Abstract. Shorter product cycles, lower prices, and the production of highly vari- 
ant products tailored to the customer needs are the main reasons for the proceeding 
success of product configuration systems. However, today’s product configura- 
tion systems are designed for solving local configuration tasks only, although the 
economic development towards webs of highly specialized solution providers de- 
mands for distributed problem solving functionality. In this paper we motivate 
the integration of several configurators and give a formal definition of the dis- 
tributed configuration task based on a logic theory of configuration. Furthermore, 
we present a basic architecture comprising several configuration agents and pro- 
pose an algorithm for cooperation between distributed configuration systems that 
ensures correctness and completeness of configuration results. 



1 Introduction 

Configurators are not only important enablers of the mass customization paradigm but 
also among the most successful applications of Al-technology. Configurators calcu- 
late product variants which fulfill customer requirements as well as technical and non- 
technical constraints on the product solution. As the digital economy of the 21®* century 
will be based on flexibly integrated webs of highly specialized solution providers, the 
joint configuration of organizationally and geographically distributed products and ser- 
vices must be supported. The rush for supply chain integration by web-based selling 
systems and electronic procurement offers new challenges for configuration technology. 
While supply chain integration of standardized, mostly well defined products can be 
quite well achieved, the case for complex configurable products and services is still an 
open research issue. 

Current configuration technology PTI| does not yet offer concepts and tools to support 
the integration of configuration systems. In particular, a distributed configuration prob- 
lem cannot be solved by a single configurator with a centralized knowledge base for 
security and privacy reasons of suppliers. As we have to cope with distributedness, we 
must accept the fact that there is no central point of knowledge. Neither a main vendor 
nor any of its suppliers has full knowledge on the whole problem domain. The second 
point is heterogeneity; we cannot assume that each of the engaged parties, that have 
knowledge on a subset of the problem domain and can therefor provide to the overall 
solution, employs the same knowledge representation formalisms. 

Consequently, our goal is to contribute to the further development of configuration tech- 
nology such that distributed configuration problems can be solved. In order to give a 
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clear and general problem definition, we base our contribution on a logical foundation. 
This abstract approach allows us to determine the basic requirements for the integration 
of special instances of configuration methods and tools found in research and industry. 
Our introductory example is based on an application scenario provided by one of our 
industry partners (Section 2). Based on the definition of a central configuration problem, 
we formally define the distributed configuration task (Section 3) and show under which 
conditions the central and distributed problems are equivalent. We present an algorithm 
which enables configuration agents to cooperatively construct valid configurations and 
describe the required properties in order to assure correctness and completeness (Sec- 
tion 4). Aspects of integration are adressed in Section 5. Finally we discuss related work 
followed by conclusions. 

2 Motivating Example 

We introduce our concepts by presenting a motivating scenario from the area of 
telecommunication systems. Our product example is a telecommunication switch for 
enterprise networks. The functionality of the switch can he extended by installing 
additional software modules onto the hardware component such as management 
software or application packages for messaging and ip-services. These additional 
applications are third-party products or may be developed by a subsidiary company. 
The customer, however, wishes to order a completely configured product solution, 
comprising the switching hardware and all needed add-ons. For obvious reasons each 
supplier maintains the product knowledge within its own sales configuration system 
that cooperates with others. In our scenario a facilitating agent coordinates the search 
for a configuration solution of three configuration agents representing the providers of 
the switching hardware, the messaging and the ipvoice application software add-ons. 
We employ a logic theory of configuration IQ that complies with the component-port 
representation for configuration knowledge H3- This logical model serves as a general 
ontology for the configuration domain. We allow that all involved configurators may use 
a proprietary representation formalism. However it must be assured that the content of 
communicated messages can be mapped onto the concepts of this logical theory. In our 
example we use only types to denote the set of component types while ports describes 
their connection points. We do not employ attributes of types and their domains in this 
example, although these concepts are part of our logic theory of configuration. 

types = {tecom, srack, hack, ipvoice, swpackl, swpack2, msger, uppack}. 

ports{tecom) = {rack, ipvoice, msger}. 

ports{lrack) = {tecom}. port s{srack) = {tecom}. 

ports(ipvoice) = {tecom, swpack}. 

ports{swpackl) = {ipvoice}. 

ports{swpack2) = {ipvoice}. 

ports{msger) = {tecom, upgr}. 

ports{uppack) = {msger}. 
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The predicates used for describing configurations are contained in a set CONL, where 
CONL = {type/2, conn/4} for our example. A type t is associated with a component 
c by literal type{c, t).A connection is represented by literal conn {cl, pi, c2,p2) where 
pi (resp. p2) is a port of component cl (resp. c2). Usually an attribute value v assigned 
to attribute a of component c is represented by a literal val{c, a, v). In our example, we 
omit val - predicates to keep the presentation short. The configuration knowledge of 
each of the three involved configurators is defined by a domain description (DD) com- 
prising sets of logical sentences that specify compatibility constraints and the derivation 
of additional facts. In addition to the constraints Cj listed below, a set of application 
independent sentences denoted by Chaste is included in the domain description, specify- 
ing that connections are symmetric, that a port can only be connected to one other port, 
and that components have a unique type. 

CID switch — {Cl, U2} U Chaste- 

ClD-ip = {U3, U4} U Chaste- ClDyyisg ~ U Chaste - 

Cl : “If the switch has more than 200 end devices then a large rack is needed. ” 

VT, C : type{T, tecom) A devices{T, C)A 
C > 200 ^ 3L : 

type{L, track) A conn{L, tecom, T, rack). 

C2 '- “If the customer requires voice-over-ip then the ipvoice application must be in- 
stalled. ” 

VT : type{T, tecom) A voice-over-ip{T) 3/ : 
type{I, ipvoice) A conn{I, tecom, T, ipvoice). 

C3 : “An ipvoice application consists either of a swpackl or swpack2 software module. ” 
VI : type{I, ipvoice) 3P : 

{type{P, swpackl) V type{P, swpack2))A 
conn{P, ipvoice, I, swpack). 

Ci : “A swpackl software module is incompatible with upgrade uppack. ” 

VT, I, M, PI, U : type{T, tecom) A type{I, ipvoice)A 
conn{I, tecom, T, ipvoice) A type{M, msger)A 
conn{M, tecom, T, msger) A type{Pl, swpackl) A 
conn{Pl, ipvoice, I, swpack) A type{U, uppack)A 
conn{U, msger, M,upgr) false. 

Ch ■ “If the software msger is sold together with the ipvoice application then it must 
contain the upgrade uppack. ” 

VT, M, I : type{T, tecom) A type{M, msger) A 
conn{M, tecom, T, msger) A type{I, ipvoice) A 
conn{I, tecom, T, ipvoice) 3T : 
type{P, uppack) A conn{P, msger, M, upgr). 
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In our domain a system requirements specification (SRS) provided by the customer is 
only sent to the switching hardware manufacturer. It is a logic theory that comprises 
predicates from CONL as well as any other predicates that specify the requirements a 
customer wants to be fulfilled. 



SRSswitch = {3T, M : type{T,tecom)A 
devices{T, 300) A voice-over -ip{T) A 
type{M, msger) A conn{M, tecom, T, msger).} 

Given the above constraints and customer requirements, central problem solving would 
achieve the following complete and consistent configuration resulfl: 

CONF = {type{idi, tecom). type{id 2 ,lrack). 
type{id^,ipvoice). type{idi, msger). 
type{id^, swpack2). type{idg, uppack).} 

Note however, that a central approach is not feasible for security and privacy concerns 
of involved business entities and the question is how to solve this task for the distributed 
case. Therefor, we aim at defining the distributed configuration problem and at stating 
the conditions under which the distributed solving generates equivalent solutions to a 
central approach. 

3 Formalizing Distributed Configuration 

In the general framework of Q, a configurator knowledge base consists of a set of logical 
sentences DD describing available component types, their attributes and connection 
points as well as constraints on legal product constellations. As sketched in Section 
2, configuration problems are solved according to a system requirements specification 
SRS and the configuration result can be described by means of a set of positive ground 
literals using predicate symbols from CONL. 

3.1 Central Configuration Approach 

Definition (Configuration problem): A configuration problem is described by a 
triple {DD, SRS, CONL), where DD and SRS are sets of logical sentences and 
CONL is a set of predicate symbols. DD represents the domain description, SRS the 
system requirements specification for a configuration problem instance. A configuration 
CONF is described by a set of positive ground literal^ whose predicate symbols are 
inCONL.U 



* For reasons of presentation, we employ only type/2 predicates for representing configurations 
and omit the conn/4 predicates 

^ By using Skolem constants we have decoupled the representation of a configuration solution 
from the problem description. Therefor validity of configurations is independent of a bijective 
renaming of these constants. 
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Definition (Consistent configuration): Given a configuration problem {DD, SRS, 
CONL), a configuration CONF is consistent iff DDL) SRS U CON F is satisfiable. □ 

To ensure the completeness of a configuration, additional formulae for each symbol in 
CONL have to be introduced to CONF, e.g., for the type predicate; 

type{X, Y) ^ type{X, Y) G CONF. 

We denote the configuration CONF extended by these axioms with CONF. 

Definition (Valid and irreducible configuration): Let {DD, SRS, CONL) be a con- 
figuration problem. A configuration CONF is valid iff DD U SRS GCONF is satisfi- 
able. CONF is irreducible if there exists no other valid configuration CON F^^^ such 
that CONF^^^^ C CONF. □ 



3.2 Distributed Configuration Approach 

Definition (Distributed configuration problem): A distributed configuration problem 
for n different configuration agents is described by a triple {DDget, SRSset, CONL) 
where 

DDset = {DDi, ..., DDn} and 

SRSset = {SRSi,...,SRSn}. 

Each element of DD set and of SRSset is a set of logical sentences and CONL is 
a set of predicate symbols. For k G DDj^ corresponds to the domain 

description of the configuration system k and SRSk specifies its system requirements. A 
configuration CONF is described by a set of positive ground literals whose predicate 
symbols are in CONL.O 

Remark: In extension to the introductory example, all configuration agents can be 
initialized with an individual system requirements specification contained in the set 

SRSset. 

Definition (Valid solution to a distributed configuration problem): Given a dis- 
tributed configuration problem {DDget, SRSset, CONL), a configuration CONF is 
valid iff DDk U SRSk U CONF is satisfiable \/k G n\.U 

In practice configurators of suppliers collaborate by exchanging (partial) configurations, 
i.e., these configurators can be seen as independent modules jointly constructing 
a common solution. Related to our case this implies that the domain descriptions 
DDi . . . DDn and the system requirements SRSi . . . SRSn of each configurator are 
independent except assertions regarding the configuration. We achieve this property by 
using disjoint sets of predicate symbols in each DDk and SRSk but allow the joint 
use of predicate symbols contained in CONL, i.e., for every pair of configurators i,j 
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{psymbol^{DDi) U psymbols{S RSi)) fl {psymbols{DDj) U psymbols{SRSj)) = 
X, where X C CONL. This way, dependencies, that surpass the knowledge of local 
companies, are considered via their effects on the configuration result. We call this 
property defined interfacing. 

Theorem: Let {DD, SRS, CONL) be a configuration problem and {DDset, SRSset-, 
CONL) a distributed configuration problem with defined interfacing where 

DD = UddGUit.et and 
SRS = srs. 

CONF is a valid configuration for {DD, SRS, CONL) iff CON F is a valid solution 
for the distributed configuration problem (DDset, SRSset, CONL).0 

Proof (sketch): 

(=>) Since DD U SRS U COWF is satishable, and DDk C DD, SRSk Q SRS also 
DDk U SRSk U CONF is satishable. It follows CONF is also a valid solution for 
the distributed configuration problem (DDset, SRSset, CONL). □ 

(<=) DDkUSRSkDCONF (we call this theory Tfc) and DDjUSRSjUCONF (called 
Tj) with k j are consistent. Let us assume DDk U SRSk U CONF U DDj U SRSj 
is inconsistent. It follows that Tk h -'(DDj U SRSj). The theory (DDj U SRSj) 
can be transformed to an equivalent theory expressed by a set of clauses Csj. 
Consequently, Tk has to imply the negation of a clause Cconl where Cconl 
follows from Csj . Note, that Tk can only imply such a clause ~^Ccon l which solely 
consists of predicates of CONL since Tk and Tj have only predicates in common 
which are in CONL. Because CONF C Tk is a complete theory w.r.t. predicates 
in CONL it follows that CONF h -<Cconl- However, DDj U SRSj implies 
Cconl and therefore Tj is inconsistent which is a contradiction to the fact that Tj 
is consistent. Consequently, DDk 0 SRSk 0 CONF U DDj U SRSj is consistent. 
By applying this argument to all elements of DDset and SRSset it follows that 
UddGDD.et dd UsrsGSflS.et CONF is consistent. □ 

Corollary: Let (DD, SRS, CONL) be a configuration problem and (DDset, SRSset, 
CONL) a distributed configuration problem with defined interfacing where 

DD = UddGDD.et dd and 
SRS = UsrsGSflS.e* 

A valid configuration CONF for (DD, SRS, CONL) is irreducible iff CONF is a 
valid solution to the distributed configuration problem (DDset, SRSset, CONL) and 
there exists no other valid solution CONF‘^'^^ to the distributed configuration problem 
such that CONF^^^ C CONF. □ 



^ The function psj/m&ois(T) returns all predicate symbols that are employed in the logical theory 
T. 
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3.3 Conflicts 



When solving a configuration problem, partial solutions are extended with the goal to 
generate valid configurations. During the problem solving phase it could be discovered 
that such partial solutions are in conflict with DD U SRS. As a consequence these 
conflicts must guide the subsequent search process in order to avoid the rediscovery of 
inconsistent configurations. Due to the defined interfacing property, conflicts can only 
be caused by predicate symbols from CONL in CONF, i.e.: 

Definition (Conflict): Let{DD, SRS, CONL) be aconfigurationproblemandCONF 
be a consistent set of sentences in CONL. CONF is a conflict of{DD, SRS, CONL) 
iff SRS U DDL -nCONF. □ 

The relation between conflicts and configurations is described as follows. 

Theorem: Let (DD, SRS, CONL) be a configuration problem, NG— CONFLICTS 
is its set of negated conflicts and CONF be a configuration including the complete- 
ness axioms. CONF is a valid configuration iff CONF U NG — CONFLICTS is 
satisfiable. □ 

Proof (sketch): 

(^) Since DD U SRS U CONF is satisfiable and NG — CONFLICTS is entailed 
by DD U SRS it follows that CONF U NG - CONFLICTS is satisfiable. □ 

( 4 =) Let CONF be a configuration where CONF U CONFLICTS is satisfiable. 
Suppose CONF is not a valid configuration i.e. DD U SRS U CONF is unsatisfiable. 
Therefore, DDL! SRS I — 'CONF. But then CONF would be a conflict and -'CONF 
must be included in AG — CONFLICTS, contradicting the fact that CONF\jNG — 
CONFLICTS is satisfiable. □ 

Definition (Minimal conflict): Let {DD, SRS, CONL) be a configuration problem 
and CONF a conflict. CONF is a minimal conflict of {DD, SRS, CONL) iff for all 
conflicts CONF^^^ either -^CONF^^^ \/ ^CONF or -^CONF^^^ = ^CONF. □ 

Note, that when searching for valid configurations we must achieve consistency with the 
negated conflicts. Since the negation of minimal conflicts implies the negation of non- 
minimal conflicts, thus a non-minimal conflict needs not to be considered. The same 
argument holds for an equivalent conflict. In the following we relate conflicts of the 
central and the distributed configuration approach. 

Corollary: Let {DD, SRS, CONL) be a configuration problem, CONF be a configu- 
ration including the completeness axioms, and {DDget, SRSset, CONL) a distributed 
configuration problem with defined interfacing where 

DD = UddSDD.et 
SRS = UsrsGSflS.e* 

CONF is a conflict for {DD, SRS, CONL) iff there exists a k € {1, . . . , n} 
CONF is a conflict for {DDk, SRSk, CONL). □ 
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Note, that every conflict found by a local configuration agent is a conflict for the com- 
plete (central) configuration problem. These conflicts must be communicated among the 
agents, in order to ensure that superfluous work on conflicting configurations is avoided. 

4 Basic Model for Interaction 

In order to show the feasibility of distributed configuration problem solving accord- 
ing to the above definitions we outline an architectural setting of cooperating agents 
and propose an algorithm for interaction. Note however, that this model represents 
a theoretical framework for the general case. For our concrete implementation in an 
application-oriented international research project we exploit domain specifics of con- 
figuration problem solving as described in the next subsection Extensions for Efficiency. 
The configuration knowledge is distributed over a set of n configuration agents which 
may configure concurrently. The communication among them is coordinated via a facili- 
tator agent that collects the (partial) configurations from each agent and distributes them 
among the others. So there is no direct communication between configurators, but only 
indirect via the facilitator. This architecture is chosen because it poses less requirements 
on the capabilities of configuration agents than peer-to-peer communication, where each 
agent must be able to distinguish between several communication channels. As soon as 
a configuration agent detects a conflict with the joint configuration, the others are in- 
formed and measures for conflict resolution are taken. This resolution strategy ensures 
that a conflict never occurs twice during a session and that the non-existence of a valid 
configuration for the overall task is detected. We do propose a very general negotiation 
strategy for conflict resolution, because we consider the integration of already existing 
configuration systems into this framework. 

The configurator agents communicate only with the faciliator, where the exchanged 
messages have the following signatures: 

- requesVff{CONF“'^)\T\\t configurator k receives the configuration CONE‘S* and 
checks if is locally satisfiable. Si denotes the search depth of the algorithm for this 
intermediate configuration solution and no counts the interaction cycles. 

- reply'^° {CON The configurator k communicates the configuration result 
CONF^''*^ in reply to requesf^° {CO N F^*) back to the facilitator. CON F^f*^ 
is a valid local configuration of configurator k. 

- conflictk{CONF^ty_ With this message the configurator k alerts the facilitator, 
that CONF^t is not satisfiable with its local knowledge base. 

- add-con flict{C) : Once the facilitator was alerted with a conflict message, it broad- 
casts this conflict C to all configuration agents that is then negated and added to 
their local system requirements SRSk- 

The facilitator agent initially distributes only the non-empty sets of individual sys- 
tem requirements SRSk to the configuration agents that store them - see Algorithm 
(a). Obviously, requesting a configuration from a configurator without any system re- 
quirements would produce merely incidental results and is therefor avoided. Then the 
facilitator starts the problem solving process by broadcasting request\{{}) to each 
recipient of a non-empty SRSk and awaits replyl{CONF^^) messages from these 
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configuration agents, only (b.2). After collection of replies (b.2) the facilitator unifies 
fhe locally completed configurations and broadcasts them to all configuration agents 
with a request'^°{[Jf. CONF^*) message. This is now possible because the intermedi- 
ate result CONF^' restricts the further solution search of agents. In case at least one of 
the remote conhgurators replies a conflictk{CONF “^ ) message the facilitator initiates 
the conflict resolution strategy (c). Here we chose a strategy where it is in the respon- 
sibility of every single agent of not delivering a conflicting configuration twice. This is 
achieved because the facilitator communicates the conflicting configuration CONF’^^ 
to all agents via an add-con flict{CON F“^) message and backtracks by demanding 
another reply to request™ {CON (c.l). All replies to the previous request are 
discarded (h.l). The algorithm terminates either with a valid solution or detects that 
there exists no solution (c.2). The latter case is implied if the empty set is not satisfiable 
with the local knowledge base including stored conflicts. A valid conhguration for the 
overall conhguration task is found, when no conhgurator has added new ground facts to 
CONF^^' during an interaction cycle (b.4), i.e., CONF^* = CONF^*+^. 

When a configuration agent receives a request™ {CON F‘‘'^) message the algorithm 
distinguishes between two problem solving levels (d). 

First the satishability of the received conhguration is tested. 



Definition (Local satisfiability): Local satisfiability for agent k is given, iff configura- 
tion CONF^* is satisfiable with its local knowledge base: DDk U SRSk U CONF’^' 
is satisfiable. 



If Local Satisfiability is given, the conhguration agent completes the initial conhguration 
CONF^^' to a locally valid conhguration CON F^^*^ (d.l), which is performed in the 
algorithm by the function configure. 



Definition (Local validity): Local validity for agent k is given, iff configuration 
CON F^'^^ is a valid configuration w.r.t. its local knowledge base: CONF^'^^ is 
valid for {DDk, SRSk, CONL). 



If Local Satisfiability is not given, the conhguration agent replies with a conflictk 
{CONF^*) message to the initial request request™ {CONF^') (d.2). 

When a conhgurator receives a add-con flict{C) message it stores it by expanding its 
local system requirements (e): 

SRS'^ = SRSk U -C. 

At this stage redundant clauses (e.g., non-minimal conhicts) can he removed from SRSk ■ 
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Algorithm - Behaviour of facilitator agent 

(a) initialize(S'i?S'set) do 

'iSRSk f=- {} : forward SRSk to agent k; 

CONF‘^° = {}; count = 1; 
ySRSk {} : send(request\{CON F‘^°)); 
end do; 

(b) when received(repl?/^°(CO-/VF^*)) do 
(b.l) if no = count then 

CONF^' = CONF^' U CONF^\- 
(b.2) t/(Vfc{G 1, . . . ,n} : received{CONF^‘))V 
{{count = 1)A 

{ySRSk ^ {} : received{CONF^^))) then 
(b.3) ifCONF^' ^ CONF^'-^ then 
count++; 

V/c G {1, . . . , n} : send{request'jf^'^*{CONF^')); 
(b.4) else 

terminate algorithm, out: CONF = CONF^'; 
end if 
end if 
end if 

end do; 

(c) when received(con//ictfc(C'0-/VF®->)) do 
(c.l) ifCONF^i ^{} then 

V/c G {1, . . . , n} : send{add-conflict{CONF^^)); 
count++; 

Vfc G {1, . . . , n} : send{request'jf^'^*{CONF^^-^)); 
(c.2) else 

terminate algorithm, out: no configuration exists; 
end if 

end do; 

Algorithm - Behaviour of configuration agent 

(d) when recei\ed(request'^°{CON F^')) do 

(d.l) if locally satis fiable{DDk U SRSk U CONF^^) then 
CONF^'^^ = configure{CONF‘^'); 
send{reply{t°{CONF^*'''^)); 

(d.2) else 

send{conflictk{CONF^')); 
end if 

end do; 

(e) when received( add-con flictk{C ) ) do 

SRSk = SRSk U -C; 

end do; 
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4.1 Extensions for Efficiency 

Configuration problems have the property to be usually underconstrained and there 
exist many good solutions that can be accepted from the standpoint of a domain expert. 
Furthermore, similar to a centralized approach, heuristics exist to guide the solution 
search within and between the configuration agents. These allow us to avoid an inefficient 
blind search of the solution space and provide optimized solutions according to some 
criteria such as price or quality. For design of our prototype implementation we identified 
the following approaches: 

- In a realistic economic setting one destined configuration agent will act as a main 
vendor that also fulfills the task of the facilitator. Further some partial sequentializa- 
tion between configurators can be assumed, i.e., the main manufacturer or service 
provider will configure locally as far as possible and restrict this way the solution 
space of its suppliers. When reducing the degree of parallelism of solution search the 
probability for conflict occurence can be obviously diminished. Restricting concur- 
rent solution search to agents whose configuration results do not have side-effects on 
each other is an additional heuristic. Further a partial ordering of configurators can 
be used for an advanced negotiation strategy for conflict resolution, where agents 
with lower priority are the first ones to repair their configuration results. Such a 
scenario where all configuration agents sequentially add new predicates to the cal- 
culated configuration of the predecessor in a supply chain is a specific instantiation 
of the more general model presented in this paper. 

- When configuring complex telecommunication systems computed configurations 
tend to become quite large with CONF encompassing thousands of facts. It is ob- 
vious that not all components and connections of the switching node are of interest 
to the configuration agent that determines the configuration of the add-on prod- 
uct. Therefor measures towards intelligent filtering of the message content need to 
be taken. We can assume that configuration knowledge is not randomly partitioned. 
Based on this partitioning of configuration knowledge, we are able to identify the vo- 
cabulary of product domain concepts that specifies the configuration capabilities and 
informational interest on components and connections for which it has constraints 
defined in its local knowledge base. By employing domain ontologies we can give 
an abstract description of the product domain of the configuration agent. Therefor 
only those facts of the overall configuration solution need to be communicated to a 
specific configuration agent that correspond to its domain ontology. 

- A further step towards lower space complexity is the reduction of conflict size. This 
can be achieved, if configuration agents are capable of generating minimal conflicts 
following the definition in Section 3.3. Techniques from model based diagnosis can 
be exploited to improve conflict generation. 



4.2 Solving the Example 

In the following we show how to solve the example from Section 2 with our algorithm. In 
the example three agents are involved, i.e., k G {switch, ip, msg}. The problem solving 
phase starts when the facilitator agent forwards the system requirements SRSswitch 
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to the switching hardware manufacturer and initiates the solution search by sending 
requestl^^f.^f^{{}) . Therefor the agent switch is the only one to reply to the facilitator 
in the first cycle of interaction: 

( 1 ) replyl^^^^f^{{type{idsi,tecom). type{ids2,lrack). 
type{ids3, ipvoice). type{ids4, msger).}) 

The facilitator distributes the received configuration as CONF^^ to all agents. 

(2) reply^^^n^f^iCONF^^ U {}) 
replyfp{CONF^^ U {type{idn, swpackl) .}) 
feplylisgiCONF^^ U {type{idrai,uppack).}) 

The facilitator unifies the received partial configurations and broadcasts a request^ 
{CONF"^^) to all agents. 

( 3 ) conflictip{CONF^^) because of Constraint C4. 

The facilitator discards the reply^ messages from agents switch and msg. It broadcasts 
add-conflict{CONF^^) and afterwards backtracks with request\{CON F^^^) to all 
agents. 

(4) replyl^^^^f^{CONF-^^ U {}) 
replyfp{CONF^^ U {type{idi2, swpack 2 ).}) 
feplytnsgiCONF’^^ U {type{idra2,uppack).}) 

The facilitator generates the union set of all received partial configurations and broadcasts 
them for another cycle of interaction. Now, no one detects a conflict. All configuration 
agents determine the validity of the configuration and do not need to derive additional 
facts. Therefor the algorithm terminates with the same solution as with central problem 
solving. 



4.3 Analysis 

For analysis we employ the basic assumption that all configuration agents are capable of 
generating valid configuration results and are complete w.r.t. the set of all valid config- 
urations, which we assume to be limited for practical reasons. This can be achieved by 
limiting the number of possible components in an artifact. In order to show the sound- 
ness of the algorithm we must show that each generated solution CONF satisfies the 
criteria of a valid distributed configuration stated in section 3 . This is given, because 
Vfc G { 1 , . . . , n} : DDk U SRSk U CONF^' is satisfiable and CONF^* = CONF. 
For proofing the completeness of the algorithm we must show that if a solution exists 
the algorithm terminates with a configuration solution CONF, otherwise the algorithm 
terminates with a failure indication. Let us first assume that the algorithm terminates. 
This happens either by giving a correct solution, or terminating, because no solution 
exists: ^CONF : DD^ U SRSk U CONF is satisfiable Vfc G n}. Finally we 

have to show the algorithm terminates. Sources for infinite processing loops are cycles 
in message passing and subsequent generation of the same conflict. Infinite processing 
loops are not possible because all agents can only receive requests from the facilitator 
that are replied either by a reply'^° or a conflictk message, and the number of these 
messages is restricted due to the initial assumption of a limited solution space. Subse- 
quent generation of the same conflict is avoided, because inconsistent configurations are 
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distributed as conflict^ to all agents via add-conflict messages and locally stored by 
them. As all valid configurations are consistent with the set of already negated conflicts, 
no conflicting configuration is produced twice hy a configuration agent. Further with this 
algorithm all valid configurations can be found. If an already found solution is negated 
and added to the system requirements, a new search can be initiated and the algorithm 
will find another solution that is different from the previous one. As there exists only a 
finite number of configurations, subsequently all solutions will be found. 



5 Aspects of Integration 

In order to be integrated in the presented architectural setting, a configurator must satisfy 
the following requirements resulting from the protocol: 

- Support of request™ requests: The configurator must accept a partial configura- 
tion as starting point for configuration problem solving and generate a complete 
configuration solution. 

- Support for add-conflict messages: The configurator must at least be able to com- 
pute alternative solutions to fulfill this requirement. An agent wrapper would then 
store the received conflicts and request alternative solutions from the native config- 
urator interface, until all stored negated conflicts are satisfied. 

The latter requirement can not be met by pure rule-based configurators, which always 
calculate only exactly one solution for given requirements. Furthermore, different ex- 
pressiveness of proprietary knowledge representations may pose a problem. Employing 
bridging rules, that map between different representation concepts are always imperfect 
in the sense that they are heuristic. 

A different issue is the process of distributedly creating and maintaining configuration 
knowledge. There must be some guidance provided and the consistency of the knowledge 
bases has to be assured. Having different representation mechanisms, a shared ontology 
must be adopted, that provides a common view on the product domain. 

Currently, separate sets of initial requirements SRSk are assumed. However, the choice 
among similar products of different companies is another point to be addressed and some 
form of reasoning on the selection of a supplier has to be introduced. 



6 Related Work 

There is a long history in developing configuration tools in knowledge-based systems. 
Progressing from rule-based systems higher level representation formalisms were de- 
veloped, such as various forms of constraint satisfaction 0|, or description logics M- 
However there is no support for integrating these systems in order to allow cooperative 
configuration. 

When using a constraint-based approach for configuration tasks, several techniques 

Note that Skolem constants in calculated configurations are converted to all-quantified variables, 
if the conflict is negated. 
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for distributed problem solving have been proposed. For problem representation a dis- 
tributed CSP is proposed in Ql and several algorithms for problem solving such as an 
asynchronous backtracking, an asynchronous weak-commitment search or a distributed 
breakout algorithm are presented. Flowever, configuration tasks are more dynamic in 
nature and therefor a CSP representation, where all problem variables must be known 
from the beginning, is not appropriate in many application domains. Dynamic constraint 
satisfaction is more suitable for representing and solving such synthesis tasks ra ca, 
because the set of problem variables may vary according to some activity contraints. 
In [l31! a distributed dynamic CSP is defined and a modification of the asynchronous 
backtracking algorithm from 111 is applied for problem solving. When configuring 
large technical systems, the limitation of a dynamic CSP representation (the amount of 
maximally active variables must be known from the beginning) becomes evident and 
a generic CSP representation 0i| j2J has been proposed. There, new instances of prob- 
lem variables can be created from meta-variables during problem solving. However, no 
representation that allows the distribution of knowledge over several agents has been 
presented so far. 

The proposed architecture for distributed configuration relates to previous research 
projects such as TSIMMIS fS] or Infomaster 0|. They provide an integrated access 
to multiple distributed heterogenous information sources on the Internet. Our approach 
for distributed configuration goes a step further, because not only information sources 
but problem-solving agents with local knowledge are integrated, thus giving the illusion 
of a centralized, homogenous configuration system. 

In the area of distributed configuration-design problem solving HD proposed an agent 
architecture. The aim of this work is to find a concurrent problem solving process in 
order to improve efficiency, whereas our concern is to provide effective support of dis- 
tributed configuration problem solving, where knowledge is already distributed between 
different agents. 

7 Conclusions 

Due to internet technologies, business processes cross enterprise boundaries, which 
boosts the demand for distributed problem solving methods. In the domain of product 
configuration the integration of Web-based configuration agents is necessary in order 
to match the needs that arise from temporary cooperation between highly specialized 
business entities. In this paper we defined a general consistency-based approach towards 
the joint provision of configuration solutions by multiple configurators. Based on a 
formal definition of the distributed configuration approach, it was shown under which 
conditions distributed configuration problem solving produces equivalent results to the 
central case. Partial configurations which are in conflict to the system requirements and 
domain descriptions facilitate the search process. The concept of conflicts was introduced 
and its relation to valid configurations shown. Further a complete and sound algorithm for 
cooperation was presented which allows the integration of domain dependent heuristics. 
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Abstract. High-level controllers that operate robots in dynamic, uncertain do- 
mains are concerned with at least two reasoning tasks dealing with the effects of 
noisy sensors and effectors: They have a) to project the effects of a candidate plan 
and b) to update their beliefs during on-line execution of a plan. In this paper, we 
show how the pGOLOG framework, which in its original form only accounted 
for the projection of high-level plans, can be extended to reason about the way 
the robot’s beliefs evolve during the on-line execution of a plan. pGOLOG, an 
extension of the high-level programming language GOLOG, allows the specifi- 
cation of probabilistic beliefs about the state of the world and the representation 
of sensors and effectors which have uncertain, probabilistic outcomes. As an ap- 
plication of belief update, we introduce belief-based programs, GOLOG-style 
programs whose tests appeal to the agent’s beliefs at execution time. 



1 Introduction 

High-level robot controllers that operate in dynamic, uncertain domains and have to cope 
with sensors and effectors that have uncertain, probabilistic outcomes are concerned with 
at least two distinct reasoning tasks. First, given a candidate plan, probabilistic projection 
allows the prediction of the effects of the plan. This task, which is a prerequisite to delib- 
erate over different possible plans, lies at the heart of probabilistic planning [KHW951 
IDHW94I and the theory of POMDPs (ICLC98I . Second, given a characterization of the 
robot’s beliefs about the state of the world, a robot should be able to update its beliefs 
during the execution of a plan interacting with the robot’s noisy sensors and effectors. 
This second task, to which we will refer to as belief update, following [BHL99I . is neces- 
sary to allow probabilistic reasoning (in particular probabilistic projection) in non-initial 
situation. 

In this paper, we show how the probabilistic high-level programming framework 
pGOLOG ||GL(X)i| , which in its original form only accounted for the probabilistic pro- 
jection of high-level plans interacting with noisy sensors and effectors, can be extended 
to reason about the way the robot’s beliefs evolve during the on-line execution of a planQ 
To do so, we hrst explicitly model a layered robot control architecture where the robot’s 
high-level controller does not directly affect the world by operating the robot’s physical 
sensors and effectors, but instead is connected to a basic-task execution level which 
provides specialized low-level processes like navigation, object recognition or grasping 

^ Here, we use the term on-line-execution in the sense of iHnrgyi . 
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objects. Having such a model has the advantage that there is a clear separation of the 
actions of the high-level controller from those of the low-level processes. In particular, 
while the action of activating a low-level process and its execution time are under the 
control of the high-level controller, neither are the effects of the activated process nor 
its completion time. 

To model the effects of the low-level processes, we make use of probabilistic pro- 
grams, where the different probabilistic branches of the programs correspond to different 
possible outcomes of the low-level processes. Modelling low-level processes as programs 
allows a very fine-grained characterization of the effects of the low-level processes at a 
level of detail involving many atomic actions, taking into account the temporal extent of 
the processes. Based on such a model of the possible effects of the low-level processes, 
we specify how the robot’s beliefs about the state of the world evolve during the on-line 
execution of a plan, in particular how the beliefs change when the robot activates a low- 
level process that operates the robot’s physical effectors or when a low-level process 
provides noisy information about the state of the world. Finally, we show how based on 
the robot’s evolving belief state it becomes possible to execute so-called belief-based 
programs, GOLOG-style programs HOLLOOl whose tests appeal to the agent’s beliefs 
at execution time. 

To get a better feel for what we are aiming at, let us consider the following ship/reject- 
example, adapted from IDHW94 II: We are given a manufacturing robot with the goal of 
having a widget painted (PA) and processed {PR). Processing widgets is accomplished 
by rejecting parts that are flawed (FL) or shipping parts that are not flawed. Initially, 
the probability of being flawed is 0.3. ship and reject always make PR True, however 
ship causes an execution error (ER) if FL holds, and reject causes ER to be True if FL 
does not hold. The robot can activate a low-level process paint, which first under-coats 
the widget (UC) for 10 seconds, then takes 20 seconds to paint it. However, paint has 
a 5% probability to fail. There is also a low-level process inspect which can be used to 
determine whether or not the widget is flawed. However, inspect has a 10% probability 
to overlook a flaw and report OK instead of OK even though the widget is flawed; if the 
widget is not flawed, it always reports OK. 

In this scenario, an example projection task is: how probable is it that the plan “first 
inspect the widget; thereafter, if OK holds then ship else reject it” will falsely ship 
a flawed widget. On the other hand, belief update is concerned with questions like: 
what is the probability that the widget is flawed if during on-line-execution the robot 
actually perceived OK (the respective probabilities are 0.3*0. 1=3% and 3/73=4.1%). 
The difference between the two tasks is that in the former case, the agent reasons about 
how the world might evolve, while in the latter case its beliefs change as a result of actual 
actions. We remark that besides updating its beliefs concerning the state of the world in 
terms of fluents like PA or PR, the robot also has to update its beliefs concerning the state 
of execution of the low-level processes; for example, 15 seconds after activation of the 
paint process the robot should not only be aware of the fact that the widget is under-coated 
by now, but also that the process is no longer in its initial state but only 15 seconds away 
from completion. Finally, a belief-based plan is a specification appealing to the robot’s 
beliefs at execution time, like for example “as long as your (i.e. the robot’s) confidence in 
whether the widget is flawed or not is below a threshold of 99%, (re-)inspect the widget. 
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Thereafter, ship the widget if your belief in the widget being not flawed exceeds 99%, 
else reject it.” Note that in this plan the activation of low-level processes is conditioned 
on the robot’s beliefs at execution time. 

The rest of this paper is organized as follows; after a brief review of the situation 
calculus and pGOLOG, we describe an overall robot control architecture for acting under 
uncertainty. Thereafter, we specify how the robot’s probabilistic belief state evolves 
during the course of action. Finally, we introduce belief-based programs and show how 
they can be used to solve the example problem. The paper ends with a discussion of 
related work and concluding remarks. 



2 The Situation Calculus 

We will only go over the situation calculus IMcCi63 LPR9KI briefly here: all terms in 
the language are of sort ordinary objects, actions, situations, or realsH There is a special 
constant S'g used to denote the initial situation, namely that situation in which no actions 
have yet occurred; there is a distinguished binary function symbol do where do{a, s) 
denotes the successor situation of s resulting from performing action a in s; relations and 
functions whose truth values vary from situation to situation are called fluents, and are 
denoted by predicate resp. function symbols taking a situation term as their last argument; 
finally, there is a special predicate Poss{a, s) used to state that action a is executable 
in situation s. Within this language, we can formulate theories which describe how the 
world changes as the result of the available actions. One possibility is a basic action 
theory of the following form tLPR98l : 

- Axioms describing the initial situation, Sq. 

- Action precondition axioms, one for each primitive action a, characterizing 
Poss{a, s). 

- Successor state axioms (SSA), one for each fluent F, stating under what conditions 
F{x, do{a, s)) holds as a function of what holds in situation s. These take the place 
of the so-called effect axioms, but also provide a solution to the frame problem. 

- Domain closure and unique name axioms for the primitive actions. 

- A collection of foundational, domain independent axioms. One of them defines how 
a situation s' can be reached from a situation s by a sequence of actions^ 

s C do{a, s') = s Q s' 
where s Q s' stands for (s C s') V (s = s'). 

Adding a Timeline. In its basic form, the situation calculus has no notion of time. 
In order to model that processes have temporal extent, we introduce a special unary 
functional fluent start of sort real. The understanding is that start(s) denotes the time 
when situation s begins (we assume that start(So) is 0). The fluent start changes its 
value only as a result of the special primitive action tUpdate{t), with the intuition that, 

^ While the reals are not normally part of the situation calculus, we need them to represent 
probabilities. For simplicity, the reals are not axiomatized and we assume their standard inter- 
pretations together with the usual operations and ordering relations. 

^ We use the convention that all free variables are implicitly universally quantified. 
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normally, every action is instantaneous, that is, the starting time of the situation after 
doing a in s is the same as the starting time of s. The only exception is tUpdateit). 
Whenever this action occurs, the starting time of the resulting situation is advanced up 
to t. The following axiom makes this precise. 

Poss{a, s) D [start{do{a, s)) = t = 

a = tUpdate{t) V t = start{s) A —dM' .a = tUpdate{t') 

We will see in Section| 5 |how tUpdate actions are used to synchronize start with the 
actual time during on-line execution of robot plans. We remark that in UGLOll a version 
of the temporal situation calculus is considered where it is possible to wait for conditions 
like a robot arriving at a certain location, which is modeled using continuous functions 
of time, an issue we ignore here for simplicity. 



3 pGOLOG 



As argued in 



robot “actions” such as paint or inspect are often best thought of 



as low-level processes with uncertain, probabilistic outcome which need to be described 
at a level of detail involving many atomic actions, rather than as primitive, atomic 
actions. To describe such processes, we proposed to model the processes as programs 
using the probabilistic language pGOLOG. The idea is to model the noisy low-level 
processes as probabilistic programs, where the different probabilistic branches of the 
programs correspond to different possible outcomes of the processes. Given a faithful 
characterization of the low-level processes in terms of pGOLOG programs, we can then 
reason about the effect of the activation of the processes through simulation of their 
corresponding pGOLOG models. 

Besides constructs such as sequences, iterations and recursive procedures, pGOLOG 
provides a probabilistic branching instruction: prob{p, ai, (T2). Its intended meaning is 
to execute program a± with probability p, and a2 with probability 1 — p. In addition to the 
constructs already present in IGLOOL we introduce the parallel construct withPol{a \ , CT2 ) 
adapted from tGLiil I . The intuition is to execute a\ and 172 concurrently until ends. 
The program tji has a higher priority than ct 2, meaning that whenever both cti and CT2 
are about to execute an action at the same time, cti takes precedence. We remark that 
pGOLOG only provides deterministic instructions. 

a primitive action 

(/)? wait/test action^ 

[ci , ct 2] sequence 

if{(j),ai,a2) conditional 

while{(j),a) loop 

prob{p, (Ti , CT2 ) probabilistic execution 

withPol{cri,a2) prioritized execution until CT2 ends 

proc{( 3 {x) a) procedure definition 



Here, a condition 0 stands for a situation calculus formula where now may be used to refer to 
the current situation; when no confusions arise, we will simply leave out the now argument from 
the fluents altogether. Similarly, the term <^[s] denotes the formula obtained by substituting the 
situation variable s for all occurrences of now in fluents appearing in 4>. 
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To illustrate the use of pGOLOG, we will now model the possible effects of paint by 
the pGOLOG program paintProc. Intuitively, if the widget is already processed, trying 
to paint it results in an error. Otherwise, 10 seconds after activation of paint the widget 
will become under-coated, and finally after 30 seconds paint will result in the widget 
being painted with probability 95 % (there is also a 5 % chance that paint will remain 
effectless). To model the effects of paint, we make use of the fluents PA, FL, PR and 
ER with the obvious meaning to represent the properties of our example domain, and 
assume successor state axioms that ensure that the truth value of PA is only affected by 
the primitive actions setPA and clipPA, whose effect is to make it True resp. False; 
similarly for the other fluents. 

procipaintProc, [waitTime{lQi) , if(PR, setER, setUC), 

waitTime{20) , if{PR, setER,prob(0.95, setPA))]). 

Here, waitTime{n) is a procedure whose purpose is to wait for n seconds. It essentially 
corresponds to a test start > r + nl, where r refers to the time where waitTime was 
invoked. We will see in the SectionElhow this and similar pGOLOG models of the robot’s 
low-level processes are used to update the robot’s beliefs during on-line execution. 

Formal Semantics. The semantics of pGOLOG is defined using a so-called transition 
semantics similar to ConGolog [GLLOOH . It is based on defining single steps of com- 
putation and, as we use a probabilistic framework, their relative probability. There is a 
function transPr{a, s, S, s') which, roughly, yields the transition probability associated 
with a given program a and situation s as well as a new situation s' that results from 
executing a’s first primitive action in s, and a new program 6 that represents what re- 
mains of a after having performed that action^ Furthermore, there is another predicate 
Final{a, s) which specifies which configurations (cr, s) are final, meaning that the com- 
putation can be considered completed. This is the case, roughly, when the remaining 
program is nil, but not if there is still a primitive action or test action to be executed. 

For space reasons, we only list a few of the axioms for transPr and Final. Let us 
first look at withPol and prob informally: the execution of ct 2 with policy a\ means that 
one action of one of the programs is performed, whereby actions which can be executed 
earlier are always preferred. If both a\ and ct 2 are about to execute an action at the same 
time, the policy takes precedence. The whole withPol construct is completed as soon 
as (72 is completed. The execution of prob{p, cti , (T 2 ) results in the execution of a dummy, 
i.e. effectless action tossHead or tossTail with probability p resp. 1 — p with remaining 
program ui, resp. 172 ■ Let nil be the empty program and a a primitive action. 

transPr{nil, s, 5, s') = 0 
transPr{a, s, S, s') = 

if Poss{a, s) A <5 = nil A s' = do{a, s) then 1 else 0 
transPr{[ai,a 2 ], s, S, s') = 

if (5 = [5', (T 2 ] then transPr{a\, s, 5' , s') 

^ Note that the use of a transition semantics necessitates the reification of programs as first order 
terms in the logical language, an issue we gloss over completely here (see IGLLOOI for details). 
For space reasons, we also completely gloss over the definition of proc, which requires a second 
order definition of transPr. 
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else if Final{ai, s) then transPr{a2, s, S, s') else 0 
transPr{prob(j>,ai,a2),s,S,s') = 

if (5 = CTi A s' = do{tossHead, s) then p else 

if (5 = (72 A s' = do{tossTail{start, s) then 1 — p else 0 
transPr{withPol{ai, (T2), s, S, s') = 

if 3 ( 5 i .(5 = withPol{6i, (T2) A transPr{ai, s, ( 5 i, s') > OA 
-<Pinal{a2) A \/S2, S2-transPr{<j2, s, 62, S2) > 0 D 
start(s') < start{s2) then transPr(CTi, s, ( 5 i, s') 
else if 3 ( 52-(5 = withPol{ai,S2)A 
transPr{a2, s, 82, s') > 0 A V^i, si. 
transPr{ai, s, Si, si) > 0 D start(s') < start(si) 
then transPr{(J2, s, 62, s') else 0 

Final{a, s) = False Final{prob{p, ai, CT2), s) = False 

Final{nil, s) = True Final{withPol{ai, 02), s) = Final{a2, s) 

So far, we have only defined which successor configurations can be reached through 
a single transition. The predicate doPr{a, s, s') defines the probability of an execution 
trace s' of program a starting in s, that is the probability to end up in a final configuration 
with situation component s' after a sequence of transitions. In the following axiom, 
transPr*{8, s, S' , s') refers to the transitive closure of transPr. 

doPr{8, s, s') = 

if 3 S' ,p.p > 0 A transPr*{8, s, S', s') = p A Final{S' , s') then p else 0 

Intuitively, if {S' , s') can be reached from {S, s), then transPr*{S, s. S', s') is the 
product of the probabilities of each transition along the path from {S, s) to {S', s'). For 
space reasons, we omit the definition of the transitive closure of transPr (which requires 
second order logic) and refer the interested reader to ICiLOOII . 



4 A Control Architecture for Acting under Uncertainty 

In modern robot control architectures like Rhino iRCF+nnl . the robot’s high-level 
controller does not directly affect the world by operating the robot’s physical sensors 
and effectors, but instead is connected to a basic-task execution level which provides 
specialized low-level processes like navigation, object recognition or grasping objects. 
We will now describe how this type of architecture can be reconstructed in a logic -based 
framework; the architecture presented here is essentially an extention of IGLOll . adapted 
to stochastic scenarios. In particular, we allow for the robot’s uncertainty about the state 
of the world, account for the fact that low-level processes have uncertain outcomes, and 
show how to deal with processes like inspect which provide information about the state 
of the world. The resulting overall architecture is illustrated in Figured] 

In order to reason about the effects of a high-level plan, we need a model of every 
part of the robot control architecture illustrated in Figured] (A robot controller that lacks 
a model of the effects of its actions is intrinsically incapable to reason about the effects 
of its actions). Let us start with a representation of the state of the world. 
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Fig. 1. Robot Control Architecture for Acting under Uncertainty 



4.1 The State of the World 

While the original situation calculus allows us to talk only about the actual state of 
the world, in scenarios like the ship/reject example we have to represent uncertain 
beliefs about the state of the world. To do so, we follow IBHL99H and characterize the 
probabilistic epistemic state of a robot by a set of situations considered possible, and 
the likelihood assigned to the different possibilities. More specihcally, there is a binary 
functional fluent p{s',s) which can be read as “in situation s, the agent thinks that 
s' is possible with weight p{s' , s)H” All weights must be non-negative and situations 
considered impossible will be given weight 0 (we do not require that the weights sum 
to 1). Furthermore, all situations considered possible in Sq must be initial. 

Vs'.p(s', Sq) > 0 D Vs", a”. s' do{a", s") 

For example, in the introductory ship/reject domain the world is in one of two states, 
si and s2, which occur with probability 0.3 and 0.7, respectively. All other situations 
have likelihood 0. The following axiom makes this precise together with what holds and 
does not hold in each of the two states. 

Vs.p(s, S'o) > 0 D ~^PA{s) A ~^PR{s) A ~^ER{s)A 
3si,S2-p{si,So) = 0.3 A p(s 2 , So) = 0.7A 
FL{si) A -iFL{s 2 ) a Vs.s Si A s S 2 D p{s, Sq) = 0 

Belief. Based onp, IBFIL99I define Bel{(f>, s), the agent’s degree of belief that f holds 
in situation s, to be an abbreviation for the following term expressible in second-order 
logic0 

^{s':0[s']}F(s', S)/Ss'p{s', S) 

Intuitively, Bel{4>, s) is the normalized sum of the weights of all situations s' con- 
sidered possible in s that fulhll f. In our example, Bel{FL, Sq) is 0.3. 

4.2 Communication between Low-Level Processes and High-Level Controller 

We assume that the entire communication between the high-level controller and the 
low-level processes is achieved through a set of registers, and model them by the special 
functional fluent reg{id, s) . The high-level interpreter can affect the value of reg by means 
of the special action send{id, val) which assigns reg{id, s) the value val. The intuition 
is that in order to activate a low-level process, the high-level controller executes a send 

® Having more than one initial situation means that Reiter’s induction axiom for situa- 
tions ILPR9HI no longer holds, just as in IBHL99II . 

’ As before, is a situation calculus formula where now is used to refer to the current situation. 
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action. For example, the execution of sendifork, paint) would tell the execution system 
to start the paint process0 

On the other hand, the low-level processes can provide the high-level controller with 
sensor information by means of the exogenou^ action reply{id,val). The following 
successor state axiom specifies how reg changes its value. 

Poss{a, s) D [reg{id, do{a, s)) = val = a = send{id, val) V a = reply{id, val)\/ 
reg{id, s) = val A ~'(3r, v.a = send{r, v) V a = replylr, z;)] 

We assume that initially, the value of the fluent reg is nil for all id, and that the robot 
know about this. 

'iid.reg{id, Sq) = nil A Vs, id.p{s, Sq) > 0 D reg{id, s) = reg{id, Sq) 

4.3 The Low-Level Execution System 

Next, let us model the low-level execution system, starting with the individual low-level 
processes. As mentioned in Section Q we model all low-level processes by probabilistic 
pGOLOG programs. While we have already modeled paint by the procedure paintProc, 
we model ship and reject by the following two pGOLOG programs. We assume that both 
processes take 10 seconds to complete execution, whereupon they confirm completion 
by means of a replyiprocessed, t) action. 

proc{shipProc, [waitTime {10) , if[PR V FL, setER), setPR, reply{processed, f)]) 

proc{rejectProc, [waitTime {10), if{PR V ~'FL, setER), setPR, reply{processed, f)]) 

Sensor Processes. Next, we turn to the process inspect. At this point, we have to explain 
what we mean by sensing. To us, sensing means: activate a sensor. This “activation” has 
as an effect a sensor reading. In the example, sensing happens through the activation of 
the inspect process, whose effect is to provide a reply{inspect, OK) or reply {inspect, OK) 
answer. We assume that the high-level controller is aware of all exogenous reply actions, 
as opposed to “actions” like setPA which are solely used to model the effects of the 
low-level processes. Sensing is thus realized by means of special low-level processes, 
which we call sensor processes and which communicate (pre-processed) sensor readings 
by means of exogenous reply actions. Note that this view of sensing significantly differs 
from the well-known sensing actions of ILev961 . 

Although during real execution the actual low-level process inspect provides the 
answer, we need a model of the behavior of the sensor to update the robot’s beliefs after 
OK or OK answers. The following pGOLOG program describes the possible effects of 
inspect. We use reply OK as an abbreviation for reply{inspect, OK) , similarly for reply OK. 

proc{inspectProc, if{F L,[wcdtTime{10) , prob{0 .9 , replyOK, replyOK)], 
[waitTime{10) , replyOK] ) ) 



* The term fork refers to the procedure fork used in UNIX- like operating systems to create new 
concurrent processes. 

® Here, an exogenous action is an action not under the control of the high-level controller. 
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Directly Observables, reply actions like the above provide the high-level controller with 
information because they assign reg{inspect, s) a value which is correlated with the value 
of FL (i.e. OK or OK) and because unlike in the case of FL there is no uncertainty about 
the value of reg. We distinguish reg from other fluents and call it directly observable, 
following tOLOO I . Directly observable fluents are such that the agent always has perfect 
information about them - like the display of one’s watch or a fuel gauge in the car. 
Formally, we call a relational fluent P directly observable wit a pGOLOG theory iff. 
the following formula holds: 

Vs, s', X. So ^ s A p{s', s) > 0 D P(s', a;) = P(s, x) 

Directly observable functional fluents are defined similarly. We remark that the initial 
and successor state axioms for reg presented in this section together with the successor 
state axiom for p in the following section guarantee that in our example reg is in fact 
directly observable. 

The Overall Low-Level Execution System. Finally, we need a formal model of the 
execution system as a whole, i.e. of the robots operating system, which ensures that send 
actions result in the activation of the corresponding low-level process. The following 
program kernelProc describes the “kernel process” of the robot’s operating system. 

proc{kernelProc , \reg(fork) ^ niU , 

if{reg(fork) = inspect, [replyifork, nil), withPol{inspectProc, kernelProc)], 
if{reg(fork) — paint, [reply(fork,nil),withPol{paintProc, kernelProc)], 

..., else [replyifork, nil), kernelProc])...)]) 

As long as regifork) is nil, nothing happens. If regifork) is assigned the name of a low- 
level process, then regifork) is reset to nil, and the low-level process is run concurrently 
to the operating system’s kernel process. We stress that pGOLOG programs such as 
the above are not intended for actual execution. Their purpose is to provide a model of 
the behavior of the low-level process. In fact, pGOLOG programs like inspectProc or 
paintProc cannot be executed by the high-level controller because it has only uncertain 
information about the value of non-observable fluents like FL, resp. because it cannot 
directly execute actions like setPA. 

4.4 The High-Level Controller 

In order to ensure that the high-level controller will always have the necessary knowledge 
to evaluate tests within high-level robot plans, we consider only a subset of the pGOLOG 
programs as legal high-level plans. This subset of pGOLOG, to which we refer to as 
GOLOG^p, consists of all programs whose tests are restricted to directly observable 
fluents, and which only execute actions that only affect directly observables. We gloss 
over the technical details. As an example, the following GOLOG^p plan activates both 
inspect and paint, waits for their completetion and finally processes the widget according 
to the result of inspect. We use OK as an abbreviation for regiinspect, OK),forkInspect 
as an abbreviation for sendifork, inspect), and similarly for forkPaint, forkShip and 
forkReject. 

prociPrgex, [forkPaint, waitTimeffOi) ,forkInspect, regiinspect) f nill , 
iffOK, forkShip, forkReject), regiprocessed) f nil?]) 
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We remark that during on-line execution of a GOLOG^p plan, whenever the high- 
level plan executes a send action, the interpreter checks whether this signals an activation 
of a low-level process and, as a side-effect, activate the actual low-level process if 
necessary. 

The Passage of Time during On-line Execution. Finally, a word on the passage of 
time during on-line execution of a high-level plan. In order to synchronise the internal 
clock, i.e. the value of the fluent start with the actual time during on-line execution, 
the high-level controller periodically generated exogenous tUpdate{t) events, where t 
refers to the actual time. As described in Section El the effect of a tUpdate is then to 
assign start the actual time. We assume that the difference A = tt+i — f between two 
subsequent updates tUpdate{ti) and tUpdate{ti+i) is smaller than the minimal delay 
between the execution time of any two actions of the pGOLOG models which have 
different execution time. Furthermore, we assume that if a reply is modeled to happen 
at time t, then during on-line execution the high-level controller will generate a tUpdate 
action causing start to advance to t before the actual reply action happens. 

5 Belief Update 

We now have a model of the robot’s control architecture, of its beliefs about the state of 
the world, and of the execution system of the robot including models of the low-level 
processes. Based on this model, we will now specify how to update the robot’s belief 
state as a result of the activation of noisy low-level processes and of the receipt of reply 
messages. We refer to this task as to (probabilistic) belief update, following IBHL99I . 

Although not quite obvious, the specification of a successor state axiom for the flu- 
ent p is not sufficent to represent the updated belief state. To see why, let us consider 
the situation Sue where the robot has activated the paint process in the initial situation 
through sendifork, paint) , after which it has waited for 15 seconds. Intuitively, the epis- 
temic state should reflect the fact that the activation of the low-level process paint has 
affected the truth value of UC. But this is not sufficient. Additionally, the robot should 
be aware of the fact that unlike in Sq, in Sue the low-level process is active, has already 
executed setUC, and is about to probably execute setPA. Thus, the paint process is no 
longer correctly characterized by paintProc, but instead by the remaining fragment of 
paintProc after 15 seconds have passed. 

The example suggests that the appropriate pGOLOG model of the low-level pro- 
cesses is not the same for all situations, but depends on the history of actions. Thus, 
we associate with every possible situation a specific pGOLOG model. Formally, we 
introduce a special functional fluent ll{s' , s) that can be read as “in situation s, the robot 
thinks that if the world is in situation s' then the low-level processes can be character- 
ized by the pGOLOG program ll{s' , s).’’ The following axiom states that in the initial 
situation the low-level processes are as described by kernelProc (defined above). 

\/s.p{s, So) D ll{s,So) = kernelProc. 

In order fo specify successor state axioms for p(s*, <fo(o, s)) and ll{s* ,do{a, s)), 
stating how the world and the low-level processes evolve from a situation s to its successor 
situation do{a, s), we have to distinguish two cases: (i) a is a reply action performed by a 
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sensor process; and (ii) a is an actions executed by the high-level controller or a tUpdate 
action. The reason that we have to distinguish reply actions from other, “ordinary” actions 
is that reply actions provide sensing information, as captured by the pGOLOG model 
of the sensing processes (like, for example, inspectProc). Note that, as stated above, 
we assume that the high-level controller is not aware of any “action” performed by the 
low-level processes except for the reply actions. 

5.1 Ordinary Actions 

Let us first consider the second case. Our solution is that the low-level processes execute 
up to the point where one of the following conditions occur: 

1 . they are blocked, i.e. waiting for a (j)l condition to become true; 

2. or they are about to execute an reply action. 

While the first condition is fairly obvious, the reason that we mind reply actions is 
that the high-level controller is aware of all reply actions, and a is no reply action. We 
will now formalize the idea to execute a program cr in s until a configuration {S, s') is 
reached where one of the above conditions is true. For this, we use the special function 
transPr^{a, s, S, s') which specifies the probability to end up in {S, s') starting in {a, s) . 
In the following formulas, lH(a) is a shorthand for 3r, v.a = reply{r, v). 

transPr^{a, s, 5, s') = 

if transPr*{(7, s, S, s') > 0 A Va*, s*.s C do{a*,s*) E s' D -'91(a*)A 
V(5*, s* .transPr{S, s', S*, s*) > 0 D 3a*. s* = do{a* , s') A iK(a*) 
then transPr*{a, s, 6, s') else 0 

While the first line of the if condition verifies thaf {S, s') can be reached from 
(cr, s) without executing any reply action, the second line verifies fhat all successor 
configurations of {6, s') can only be reached by a violation of the second of the above 
conditions, meaning that the simulation has been pursued as far as possible. 

Using transPr^, we can define which configurations (s*, IP) have been reached 
by the low-level processes in do{a, s) together with their weight (assuming that a is no 
reply action). Intuitively, these are all configuration that result from the execution via 
transPr^ of a configuration {ll{s' , s), s') considered possible in s. Their weight is the 
product of the weight of s' in s and the transition probability as specified by transPr^. 
The predicafe advConfig{s* , IP ,do{a, s)) makes this precise. 

advConfig{s* ,IP ,do{a,s)) = p = 

3s' , p' , p* .p{s' , s) = p' A transPr^{ll{s' , s), s', lP,s*) = p* A 
p' > 0 A p* > 0 A p = p' • p*V 

p = 0 A -'3s'.p(s', s) > 0 A transPr^{ll{s' , s), s', IP, s*) > 0 



5.2 reply Actions 

Now that we have formalized how the low-level processes evolve if a is an ordinary 
action, let us turn to the other case where a is a reply action. Intuitively, the observation 
of a reply should sharpen the belief state of the robot. For example, if the robot observes 
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a replyOK action after activation of inspect, it can rule out those situations from its 
belief state where ^FL holds. In general, the observation of a reply action can be used to 
rule out those situations whose associated pGOLOG model of the low-level processes 
ll is not about to execute this very reply action. To make this precise, we define the 
predicate adv&filter{s*,ll*,do{a,s)) which - if a is an reply action - preserves only 
those configurations of advConfig whose pGOLOG-component is about to execute a. 

adv&filter{s* ,11* , do{a, s)) = p = 3s'. s* = do{a, s')A 
[-■ 111 ( 0 ) A advConfig{s' , ll* ,do{a, s)) = pV 
lH(a) A [3s", ll" ,p" ,p* .advConfig{s” , ll" , do{a, s)) = p"A 

transPr* {ll" , s" ,11* , s*) = p* A p" > 0 A p* > 0 A p = p" ■ p*V 
p = 0 A 9f(o) A -'3s", ll" .{advConfig{s" , ll" , do{a, s)) > OA 
transPr*{ll",s",ll*,s*) > 0)]] 

If a is an ordinary action, adv&filter is almost like advConfig-, the only difference 
is that all situations s* considered in do{a, s) now “end” with action a, i.e. 3s'. s* = 
do{a, s'). However, if a is a reply action, then we keep only those situations s* in the 
belief state whose associated pGOLOG model correctly predicted that the reply action 
a would be executed next. 

Successor State Axioms for p and ll. It can be shown that the function adv&filter is 
well-defined, meaning that any configuration (s*, ll*) with positive weight is assigned 
exactly one weight. Furthermore, it can be shown that for each situation s* there is at 
most one ll* such that adv&filter{s* , ll* ,do{a, s)) > 0. Therefore, p and ll can simply 
be defined as the situation resp. pGOLOG component of adv&filter. 

p{s*,do{a, s)) = p = 311* ,adv&filter{s* , ll*, do{a, s)) = pA 
p > 0 V MU* ,adv&filter{s* ,11* ,do{a, s)) = 0 A p = 0 

ll{s* ,do{a, s)) = ll* = adv&filter{s* , ll* , do{a, s)) > OV 
MU' .adv&filter{s* ,IV , do{a, s)) = 0 A ll* = nil 



5.3 Examples 

To illustrate how p and ll evolve, and in particular how the perception of an exogeneous 
reply action is used to sharpen the robot’s beliefs, we will now consider the value of p 
and ll in different situations. We begin with situation S'inspect = do{[send(fork, inspect), 
replyifork, nil), tUpdate{l), ..., tUpdate{10)], Sq), already mentioned above. Let P be 
the foundational axioms of Section21(except for the induction axiom) together with the 
successor state axioms for p and ll, action precondition axioms stating that all set and 
clip actions are always possible, successor state axioms for the fluents PA, FL, PR and 
ER, and the probabilistic characterization of the initial state of Section El Then, from 
P we can deduce that in ^inspect two situations are considered possible. Intuitively, the 
first one corresponds to the case where the widget is flawed and the second one to the 
case where it is not flawed. Furthermore, we can deduce that these situations have an 
associated pGOLOG model of the low-level processes that accounts for the fact that the 
paint process is active and about to provide a reply. 
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r ^ Vs', lV.p{s', ^inspect) > 0 A II {s' , ^inspect) = H' = 

3s* .s' = do{[send(fork, inspect), replyifork, nil), ..., tUpdate{10)], s'q) 

A[ll' = conc{kernelProc, [start > 101 ,prob{0.9, replyOK, replyOK)])\/ 

IV = concikernelProc, [start > 101 , replyOK])] 

We remark that so far the robot’s belief concerning the value of FL remains unchanged 
(_r 1= Bel{FL, 5'inspect) = Bel{FL, Sq)). Now assume that the inspect process provides 
a replyOK answer, leading to Situation S'-.ofc = do{replyOK, Siaapect) ■ Intuitively, we 
would expect that after this observation the robot no longer considers a situation possible 
where the widget is not flawed. Indeed, we can deduce that in S'-.ofc the robot only 
considers one situation possible, and that FL holds in this situation. 

r ^ p{s',S^ok) = p Ap > 0 = 3s'q. p(s'q,S'o) > 0 a 

s' = do{[send{fork, inspect), replyifork, nil), ..., tUpdatefO), 
tossFlead, replyOK] , s'q) A FL(s') A FL(s'q) 

Intuitively, the only situation that remains in the belief state corresponds to the 
simulation trace where FL holds, and inspect correctly reports replyOK. This corresponds 
to the execution of the first branch of the prob instruction in the pGOLOG model 
inspectProc, leading to a tossHead action in the resulting execution trace. All other 
simulation traces would end up in a replyOK answer, and are thus ruled out from the belief 
state by adv&filter. We remark that the resulting belief state implies Bel{FL, S^ok) = 1- 

Similarly, if the robot would observe replyOK, we could deduce that only two situa- 
tions are considered possible in the resulting situation dofeplyOK, 5'inspect): one corre- 
sponding to the widget beeing flawed (prob. 30%) and inspect erroneously reporting OK 
(prob. 10%), and another one where the widget is not flawed (prob. 70%). The robot’s 
resulting belief in FL would then correspond to the normalized probability of the first 
case, which is (0.3 * 0.1)/ (0.7 + 0.3 * 0.1) = 

As another example, let us consider the situation Suc= do{[send fork, paint) , ..., 
tUpdatefb)] , 5g), where the paint process is active for 15 seconds. In this situation, the 
low-level process paint has already caused UC to become true, and is waiting until time 
30 whereat it may cause PA to become true. 

r h Sue) > 0 = 

3s*. s' = do{[send fork, paint), ...,tUpdateilO),setUC, 
tUpdatefl) , ..., tUpdatefb)], s*)A 

Ilf , Sue) = concfernelProc, [start > 30?, if{PR, setER, probiO.95, setPA))]) 

Some seconds later, the robot’s belief in PA will raise to 95% due to the fact that it 
will assume that paint has finished execution. However, the robot’s belief in the widget 
beeing flawed will remain unchanged. 



6 Belief-Based Programming 

As an application of belief update, we will now introduce the concept of belief-based 
programs, GOLOG^p programs that appeal to the robot’s beliefs at execution timej^In 



This is similar to Reiter’s notion of knowledge-based programming fReiPOI . However, we 
remark that here we are dealing with degrees of belief. 
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particular, we introduce a special epistemic test BTest{(p, p,s), which is true if in situation 
s the robot’s belief in (f) is p. Formally, BTest{(f>, p, s) is a defined relational fluent which is 
true iff. Bel{cj), s) = p. Using BTest within test conditions, a GOLOG^p plan can appeal 
to the robot’s beliefs at execution time. As an example, the following plan specifies that 
the robot is to activate the inspect process until it is sufficiently confident about whether 
the widget is flawed or not. Thereafter, the widget is painted and processed^ 

proc{savePaint, 

[ while{3p.BTest{FL,p) Ap> 0.001 Ap < 1, 
send{inspect,nU),forkInspect,reg(inspect) ^ nil?), 
forkPaint, waitTime{30) , 

if{BTest{FL, l),forkRejectJorkShip), regiprocessed) ^ nill] 

We remark that the above program causes at most three activations of inspect: as we 
have seen in the previous section, the observation of one OK answer causes the robot’s 
belief in FL to drop to 3/73. Similarly, the observation of three OKs causes the robot’s 
belief to drop to (0.3 * 0.1 * 0.1 * 0.1) /(0.7 + 0.3 * 0.1 * 0.1 * 0.1), which is less than 
0.001 . On the other hand, the observation of a OK answer immediately causes the robot’s 
belief in FL to rise to 1 . 

Unlike ordinary GOLOG programs which are conditioned on facts about the world, 
in belief-based programs like the above actions are conditioned on the robot’s belief 
state at execution time. As the example illustrates, belief-based programs allow the 
programmer to provide domain dependent procedural knowledge in a natural way. From 
a pragmatic point of view, belief-based programming can be an attractive alternative 
to probabilistic planning because it represents a much simpler computational problem. 
While probabilistic planners are searching for an (optimal) plan from first principles, 
which in the worst case means that an exponential number of candidate plans has to be 
projected, the execution of a belief-based program only requires the computation of the 
belief state of the robot along the execution of a given plan. 

Implementation. Just as in the case of ConGolog, it is straightforward to implement 
a pGOLOG interpreter in PROLOG. We remark that our implementation was able to 
execute the above belief-based plan in a fraction of a second. 



7 Discussion 

Summarizing, we have shown how to update the probabilistic belief state of a robot during 
on-line execution of high-level GOLOG^p plans. To do so, we have modeled a layered 
robot control architecture within the pGOLOG framework, making use of probabilistic 
pGOLOG programs to model noisy low-level processes. In order to deal with sensing, 
we have introduced the concept of sensor processes, low-level processes whose activation 
results in exogenous reply actions. Finally, we have introduced belief-based programs, 
GOLOGrp programs whose tests appeal to the agent’s beliefs at execution time. We 
remark that unlike approaches like ILev9blBHL991 , we represent the belief state of the 
agent by a set of possible situations and an associated model of the state of execution of 

As usual, we leave out the now argument in the tests, in particular in the epistemic fluent BTest. 
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the low-level processes, which allows us to account for noisy processes with temporal 
extent. 

The whole framework, in particular the definition of p and ll, relies on the fact 
that pGOLOG programs are deterministic. As a result, it is not possible to specify 
unprioritized concurrency as done in ConGolog where the resulting course of actions is 
not uniquely determined. However, when we consider processes with temporal extent, 
this does not seem to be a severe restriction, because the priority of a process manifests 
only when two processes wish to execute an action at exactly the same time; actions 
with different execution times are not affected. 

Probably the closest work to that reported in this paper is that of Bacchus, Halpern 
and Levesque 1BHL99I . to which we owe the characterization of the robot’s epistemic 
state. However, while we manage solely with the prob instruction to represent noise, 
they make use of the concepts of nondeterministic instructions, action-likelihood ax- 
ioms OI{a, a' , s) and observation-indistinguishability axioms l{a, s), and represent the 
execution of noisy actions as atomic. This results in a simpler SSA for p, but at the 
cost of a more complex specification of the effects of the noisy sensors and effectors. 
Furthermore, it is not clear how to project a plan within their framework. On the other 
hand, probabilistic projection in the pGOLOG framework was already considered in 
I GLOOM , and it would be relatively straightforward to consider both projection and belief 
update within pGOLOG. 

As for probabilistic planners like C-Buridan [DHW941 . they usually completely 
ignore belief update. Besides, they represent processes as atomic actions. The latter also 
holds for the theory of POMDPs (which is concerned with both reasoning tasks), but 
whose computational cost is prohibitive already in relatively small domains. We believe 
that in many domains, the use of belief-based programs providing procedural knowledge 
is more promissing than uninformed search for an optimal plan. In IIPoo98l . Poole 
proposes an integration of decision theory and the situation calculus, which however is 
primarily concerned with the expected utility of a candidate plan. Finally, the recently 
proposed DTGolog IBRSTOOl assumes full observability of the domain. All of these 
approaches do not account for the temporal extent of the low-level processes. 
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Abstract. We present solutions of benchmark instances to the solitaire 
computer game Atomix found with different heuristic search methods. 
The problem is PSPACE-complete. An implementation of the heuristic 
algorithm A* is presented that needs no priority queue, thereby hav- 
ing very low memory overhead. The limited memory algorithm IDA* 
is handicapped by the fact that, due to move transpositions, duplicates 
appear very frequently in the problem space; several schemes of using 
memory to mitigate this weakness are explored, among those, “partial” 
schemes which trade memory savings for a small probability of not find- 
ing an optimal solution. Even though the underlying search graph is 
directed, backward search is shown to be viable, since the branching 
factor can be proven to be the same as for forward search. 



1 Introduction 

Atomix was invented in 1990 by Gunter Kramer and first published by Thalion 
Software for the popular computer systems of that time. The goal is to assemble 
a given molecule from atoms (see Fig.OJ. The player can select an atom at a time 
and “push” it towards one of the four directions north, south, west, and east; it 
will keep on moving until it hits an obstacle or another atom. The game is won 
when the atoms form the same constellation (the “molecule” ) as depicted beside 
the board. A concrete Atomix problem, given by the original atom positions and 
the goal molecule, is called a level of Atomix. 

The original game had a time limit and did not count the moves needed; 
we will instead focus on the analytical aspect and try to minimize the solution 
length as a goal. Note that we are only interested in optimal solutions; in order 
to just find any solution fast, quite different algorithms would be necessary. 

An implementation of this Atomix variation for the X Window System is 
available as “katomic” from http://gcmies.kde.org. A JavaScript version can 
be played online at http : / /www . sect . mce . hw . ac . uk/~peteri/ atomix. 

Our solver program written in C-|— I- is able to solve 17 of the 30 problems 
from the original Atomix and 18 of the 67 problems from katomic optimally. In 
an appendix, we list a selection of these findings. 



F. Baader, G. Brewka, and T. Eiter (Eds.): KI 2001, LNAI 2174, pp. 229-^4^ 2001. 
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Fig. 1. Two Atomix problems. The left one — which is Atomix 01 in the list of the 
appendix — can be solved with the following 13 moves, where the atoms are numbered 
left-to-right in the molecule: 1 down left, 3 left down right up right down left down 
right, 2 down, 1 right. The right one — which is level number 43 from the “katomic” 
implementation — illustrates a more complex problem; it takes at least 66 moves to 
solve. 



2 Heuristic Search 



Many common problems and, especially, most solitaire puzzles can be formulated 
as a state space search problem: given are a start state, a set of goal states and 
a set of operators to transform one state into another; wanted is a sequence of 
operators, also simply called a move sequence, that transforms the start state 
into a goal state and that is of minimal length. A state space can be represented 
as a graph, with nodes representing states and (directed) edges representing 
moves. That way, well-known graph algorithms can be applied. To emphasize 
this aspect, states generated in a state space search are often called “nodes”. 

For hard combinatorial problems, the use of heuristics can often lead to 
dramatic improvements for a state space search. Many problems would even be 
unsolvable without them. For a state space search, “heuristic” has a well-defined 
meaning: an estimate of the moves left from the current state to a goal. Of 
special interest are admissible heuristics: they never overestimate the number of 
moves. The well-known algorithms A* and IDA* can be proven to always find 
an optimal solution when using an admissible heuristic. An admissible heuristic 
judges the “quality” of a state s: if g{s) is the number of moves already applied, 
and h{s) is the heuristic estimate, then /(s) := g{s) + h{s) is a lower bound on 
the total number of moves. This number, customarily called the “/-value” , can 
be used in two ways: to guide the search and to reduce the effective depth of the 
search. The first idea naturally leads to the A* algorithm: “promising” states are 
examined first. The second is applied in the IDA* algorithm: “hopeless” states 
are not examined at all. 
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3 Related Puzzles 

The following table compares some search space properties of Atomix to other 
games. The results are contained in mm- 

Table 1. Search space properties of some puzzles. The effective branching factor is 
the number of children of a state, after applying memory-bounded pruning methods 
(in particular, not utilizing transposition tables; see Sect, lb. ill for the methods applied 
to Atomix). For Sokoban and Atomix, the numbers are for typical puzzles from the 
human-made test sets; for Sokoban, those problems are about 20x20 and, for Atomix, 
about 16x16 squares large. 



24-Puzzle Sokoban Atomix 



Branching factor 


2-4 


0-50 


12-40 


effective 


2.3 


10 


7 


Solution Length 


80-112 


97-674 


8-120 


typical 


100 


260 


45 


Search space size 


1025 


IQi® 


10^^ 



Graph Undirected Directed Directed 



Due to its close relationships to Atomix (which will become important in the 
next section), we discuss the 15- and the 24-puzzle as special instances of the 
(n^ — l)-puzzle in more details. 

The 15-puzzle consists of a square tray of size 4x4 with 15 tiles numbered 1 
through 15 and one empty square. A move consists of sliding one tile adjacent 
to the empty square into the empty space. The goal is to obtain the usual 
ordering of the numbers on the tiles by some move sequence. The 15-puzzle is 
likely to be the most thoroughly analyzed puzzle of this kind [SI- It serves as a 
kind of “fruit fly” for heuristic search. It is easy to implement, has an obvious 
heuristic with the “Manhattan distance”, and not too large a search space. The 
Manhattan distance heuristic can be calculated by summing up the number of 
turns it would take for a tile to get to its goal position if it was the only tile in 
the tray. This is obviously a lower bound on the actual number of turns. 

Many search methods developed for the 15-puzzle can be easily adapted for 
Atomix. One important difference is that the underlying search graph for Atomix 
is directed; not every move can be undone. 

Improved heuristics for the 15-puzzle make it possible to solve even the ex- 
tended “24-puzzle” -variation US]. Most of them follow the common theme of 
examining a sub-problem where only a few tiles are regarded and most are ig- 
nored. The “linear conflict heuristic” 0, for example, tries to And pairs of tiles 
in a row or column which need to pass each other to get to the goal position. In 
such a case, another two moves can be added to the heuristic given by the Man- 
hattan distance, since one tile will have to move out of the way and back. The 
work of Culberson and Schaeffer |2| generalizes this idea to “pattern databases” : 
Each possible distribution of the tiles 1-8 on the board is analyzed and solved, 
yielding a lower bound which is often better than the Manhattan heuristic with 
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Fig. 2. The left problem can be solved in 13 moves. We cannot get a lower bound by 
leaving out one atom, as in the right picture; the problem even becomes unsolvable. 



the linear conflict heuristic, since there are more tile interactions. The same is 
done for the other 7 tiles. Unfortunately, these powerful techniques cannot be di- 
rectly applied to Atomix, since removing atoms from a state does not necessarily 
make it easier to solve; in fact, it can even become unsolvable, see Fig.|3 

4 Complexity of Atomix 

4.1 Complexity of Sliding-Block Puzzles 

The time complexity of sliding block puzzles was the subject of intense research 
in the past. Though seemingly trivial, most variations are at least NP-hard and, 
some, even PSPACE-complete. The following table shows some results. The table 
was basically taken from Demaine et al. |S| , extended by the category of games 
where the blocks are pushed by an external agent not represented on the board, 
into which Atomix falls. The columns mean: 

1. Are the moves performed by a robot on the board, or by an outside agent? 

2. Can the robot pull as well as push? 

3. Does each block occupy a unit square, or may there be larger blocks? 

4. Are there fixed blocks, or are all blocks movable? 

5. How many blocks can be pushed at a time? 

6. Does it suffice to move the robot/a special block to a certain target location, 
instead of pushing all blocks into their goal locations? 

7. Will the blocks “keep sliding” when pushed until they hit an obstacle? 

8. The dimension of the puzzle: is it 2D or 3D? 



Game 


1. 2. 

Robot Pull 


3. 

Blocks 


4. 5. 6. 7. 8. 

Fixed # Path Slide Dim. 


9. 

Complexity 


PushPush3D 


-t 


unit 


— 


1 


+ 


-f 


3D 


NP-hard 


PushPush 


+ 


unit 


— 


1 


+ 


-b 


2D 


NP-hard 


Push-* 


-f 


unit 


— 


k 


— 


— 


2D 


NP-hard 


Sokoban-I- 


-f 


1x2 


-f 


2 


— 


— 


2D 


PSPACE-compl. 


Sokoban 


-f 


unit 


-f 


1 


— 


— 


2D 


PSPACE-compl. p 


15-Puzzle 


— 


unit 


— 


1 


— 


— 


2D 


NP-compl. 


Rush Hour 


- 


lx{2,3} 


- 


1 


+ 


- 


2D 


PSPACE-compl. 


Atomix 


- 


unit 


-f 


1 


- 


-b 


2D 


PSPACE-compl. [ 1 Z) 



Finding Optimal Solutions to Atomix 233 

4.2 A Formal Definition of Atomix 

We will now give a formal definition of an Atomix problem instance (level). 
Definition 1. An Atomix problem instance consists of: 

— A finite set A of so-called atom types. 

— A game board B = {0, . . . , w — 1} x {0, . . . , — 1}. 

— A bit matrix O = {0[p\ G {0,1} | p G B) of size wxh (the obstacles}. A 
position is simply a tuple p = (px,Py) G B. A state s is defined as a subset 
of Ax B. An element of s is also called an atom. Note that the same atom 
type might appear several times in a state. 

A position p = (px,Py) is said to be empty for a state s ifO[p] = 0 and there 
is no a ^ A with (a, (PxjPy)) G s. 

Positions outside of B areassumed not to be empty. 

— A state S (the start state}, which satisfies that, for all (a,p) G 5', 0[p] = 0. 

— A state G (the goal state}. For the problem to be solvable, for all (a,p) G G, 
0[p] = 0 and there must be a bijection between S and G where each atom in 
S maps onto an atom in G with the same atom type. 

A direction (dx,dy) is a tuple of x and y offsets, i. e., one o/(0, — 1), (1,0), 
(0,1) and (—1,0). A move is a tuple of a position p and a direction d. For a 
state s, a move (p,d) is only legal if there is an atom (a,p) in s, and (px + 
dx,Py + dy) is empty. 

Applying a move (p, d) to a state s will yield another state s' in which every 
atom has the same position, except the atom (a,p): it will be replaced by {a,p') 
with p' = {px + 6dx,Py + 5dy), where {px + S'dx,Py + S'dy) is empty for all 
0 < 6' < S, and [px + (5 + l)dx,Py + (<5 + l)dy) is not empty. A solution is a 
sequence of moves which, incrementally applied to the start state, yields the goal 
state. 

The main difference between this formal definition and the informal intro- 
duction is that the goal positions of the atoms are given explicitly. The reason 
is that this makes the puzzle both easier to analyze and to implement. Since the 
number of goal positions is linear in the board size, this difference does not affect 
the time complexity significantly. Our implementation handles different possible 
goal positions by imposing a move limit and trying all possible goal positions 
with that limit, and repeating with an incremented move limit until a solution 
is found 0 



4.3 The Hardness of Atomix 

Proposition 1. Atomix on an nxn board is NP-hard. 



^ As explained later, this incremental approach is already inherent to IDA*, and can 
be applied to A* with reasonable overhead. 
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Proof. Any {vf — l)-puzzle instance can be transformed into an Atomix in- 
stance by replacing the numbered tiles with atoms of unique atom types. For 
the {nf — l)-puzzle, a legal move consists of sliding a tile into the empty space. In 
the reduction, those are also the only legal moves, since all atoms not adjacent to 
the empty square cannot satisfy the move legality condition, and those adjacent 
to the empty square can only take its place as a move. As shown by Ratner and 
Warmuth, the {nf — l)-puzzle is NP-complete so Atomix is NP-hard. □ 



Proposition 2. Atomix on an nxn board is in PSPACE. 

Proof. A nondeterministic Turing-machine can solve Atomix by repeatedly ap- 
plying a legal move from the start state encoded on its tape until a goal is 
reached. The number of possible Atomix states is limited by n^l; hence, the 
machine can announce that the puzzle is unsolvable after having applied more 
moves without finding a solution. Since an encoding of an Atomix state needs 
only polynomial space, it follows that Atomix is in NPSPACE = PSPACE. □ 

Very recently, Holzer and Schwoon H2| showed by reduction from non-empty 
intersection of finite automata that Atomix is even PSPACE-complete. They 
also provide a level with an exponentially long optimal solution. 



5 Searching the State Space of Atomix 

Much progress has been made in the area of heuristic search. This is due to: faster 
machines with more memory, better heuristics, and better search methods. Of 
these three, by far, the largest improvements come from better heuristics. 

5.1 Heuristics for Atomix 

As is often the case, a heuristic for Atomix can be devised by examining a model 
with relaxed restrictions. We drop the condition that an atom slides as far as 
possible: it may stop at any closer position. These moves are called generalized 
moves^ In order to obtain an easily computable heuristic, we also allow that an 
atom may also pass through other atoms or share a place with another atom. 
The goal distance in this model can be summed up for all atoms to yield an 
admissible heuristic for the original problem. 

The following properties are immediate consequences of the definition. 

Property 1. The heuristic is admissible. 



Property 2. The /i-values of child states can only differ from that of the parent 
state by 0, -1-1 or —1. 



^ The variant of Atomix which uses generalized moves has an undirected search graph. 
Atomix with generalized moves on an nx n board is also NP-hard but is in PSPACE. 
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Property 3. The heuristic is monotone (consistent), i.e., the /-value of a child 
state cannot be lower than the /-value of the parent state. 

Apart from this somewhat obvious heuristic, it proved to be pretty hard to 
make any improvements. Two ideas were considered, but not implemented due 
to their limited applicability: 

If an atom needs a “stopper” at a certain position to make a turn for each 
optimal path, but no optimal path of any atom has an intermediate position at 
the stopper position, h can be incremented by one. 

If an atom is alone in a “cave”, for some positions, one or two moves can 
be added to the heuristic (see the example below). A “cave” is an area that 
contains no goal position and has only one entry; if an atom is alone in there, 
it cannot use any stoppers unless another atom leaves its optimal path. This 
heuristic has a greater potential, since it can be added up admissibly for each 
cave. Unfortunately, only a few levels from our test set contain caves which could 
yield improved heuristics. 




Fig. 3. An example for the “cave” -heuristic: if 
only one atom is in the cave, the number de- 
noted on its square can be added to the heuris- 
tic estimate. For example, an atom on the light 
grey square has to take the path marked with a 
solid line, instead of the optimal path of gener- 
alized moves marked with a dashed line, which 
is two moves shorter. 



5.2 A* 

A* is one of the oldest heuristic search algorithms m- It is very time-efficient, 
but needs an exponential amount of memory. A* remembers all states ever en- 
countered, which is the reason for its exponential space complexity. A priority 
queue holds all states that have not yet been expanded. It is sorted by the /- 
value of the states. Nodes are popped from the queue and expanded afterwards. 
The children are inserted into the queue or discarded if they were already en- 
countered. Sometimes, the same state is reached with a lower g-value; in that 
case, its entry in the state table has to be updated and it will be re-inserted into 
the queue. With an admissible heuristic. A* will always find an optimal solution. 

The state table is usually implemented as a hash table for fast access and 
low memory overhead. The priority queue can be implemented with a bucket 
for each /-value, containing all open states with that /-value. In Sect. Ib.2| we 
present an alternative implementation that only needs the state table and does 
without a priority queue. 
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5.3 IDA* 

Iterative Deepening A* (IDA*) (see ^5) was the first algorithm that allowed 
finding optimal solutions to the 15-puzzle. IDA* performs a series of depth-first 
searches, with an increasing move limit. The heuristic is used to prune subtrees 
where it is known that the bound will be exceeded, since the /-value is larger 
than the bound. Each iteration will visit all nodes encountered in the previous 
iteration again; but, since the majority of nodes will be generated in the last 
iteration, this does not affect the time complexity. 

IDA* uses no memory except for the stack, so its memory use is linear in 
the search depth. Also, since it needs no intricate data structures, it can be 
implemented very efficiently. But of course, this comes at a price: IDA* does 
not detect transpositions in the search graph. If a state is encountered that 
has already been expanded and dismissed, it will be expanded again, possibly 
resulting in the re-evaluation of a huge subtree. There are two approaches to 
lessen this weakness: use of problem specific knowledge and use of memory. 

Pruning the Search Space. Several techniques are known for pruning, e. g., pre- 
decessor elimination, which disallows to take back moves immediately. For games 
with undirected underlying graphs like the 15-puzzle, this is an obvious optimiza- 
tion. For Atomix, it can still be applied, since pushing an atom into the opposite 
direction immediately after a move always yields the same state as pushing it in 
that direction in the first place. 

Move Pruning. When examining a solution move sequence for an Atomix level, 
one notices that many, though not all moves could be interchanged. Interchang- 
ing moves is not possible in four cases, as is explained in Fig. ^ 




Fig. 4. There are four cases 
where two moves are de- 
pendent, i.e., their order 
cannot be interchanged: (1) 
The current atom would 
have stopped the previ- 
ously moved atom earlier. 
(2) The current atom uses 
the previously moved atom 
as a stopper. (3) The cur- 
rent atom wonld stop ear- 
lier if the previously moved 
atom had not been moved. 
(4) The current atom was 
the stopper of the previ- 
ously moved atom. 



The idea is to check if a generated move is independent of the previous 
move (i.e., applying them in reversed order would yield the same state) and. 
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if they are independent, to impose an arbitrary order (the atom with the lower 
number must move first). This scheme has proven to be very efficient in avoiding 
transpositions, reducing running time by several orders of magnitudes. 



5.4 Partial IDA* 

Analogously to the two-player game search, a transposition table can be used 
to avoid re-expanding states CHI States are inserted into a hash table together 
with their g-value as they are generated. Then, for each newly generated state, 
it is looked up whether it has already been expanded with the same or a lower g- 
value so it can be pruned. If memory was unlimited, this would avoid all possible 
transpositions. Many schemes have been proposed for proper management of the 
transposition table with limited memory our implementation simply refuses 
to insert states into an exhausted table. 

A lot of memory can be saved with Partial IDA * |tlYj . This idea originates in 
the field of protocol verification, where the objective is to generate all reachable 
states and check if they fulfill a certain criterion. A hash table is used to avoid 
re-expanding states. Just as for a single-agent search, memory is the limiting 
resource. Therefore, Holzmann suggested bitstate hashing m]: instead of storing 
the complete state, only a single bit corresponding to the hash value is set, 
indicating that this state has been visited before. Because of the possibility of 
hash collisions, states might get pruned erroneously, so this method can give false 
positives. When applied to IDA*, states on optimal paths could get pruned, so 
the method looses admissibility, but is still useful to determine upper bounds 
and likely lower bounds. 

For Atomix, initial experiments with Partial IDA* rarely found optimal so- 
lutions. The reason is that just knowing a state has been encountered before is 
not sufficient, because if we encounter it with a lower (7-value than previously, it 
needs to be expanded again. To achieve this, we include g into the hash value 
and look up with g and 5 — 1. This means transpositions with better g will not 
be found in the table and expanded, as desired. Transpositions with g worse by 
2 or more will also not be detected; experiments showed that they are rare and 
the resulting subtrees are shallow, though. 

By probing twice (with g and (7 — 1), we increase the likelihood of hash 
collisions. For example, if we declare the table to be full if every 8th bit is set, 
we have an effective memory usage of 1 byte per state, and a collision probability 
of 1— (I) = 23%. To improve the collision resistance, one can calculate a second 
hash value and always set and check two bits, effectively doubling memory usage 
but lowering collision probability to 1 — (||) = 3%. 

A related scheme with better memory efficiency and collision resistance is 
hash compaction m- It utilizes a hash table where, instead of the complete 
state, only a hash signature is saved. In our implementation, we use 1 byte for 
the signature, and probe for g and g — 1. This way, we have a collision probability 
of 1 — (III) = 0.8%, so even if there is only a single possible solution of length 
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30, the probability of finding it is 




= 79%; and in fact, all 47 solutions 



found this way were optimal. 

Different policies are possible in the case of a hash collision detected by 
differing signatures. Usual hash table techniques like chaining or open addressing 
can be applied. We tried a much simpler scheme: the old entry gets overwritten. 
This can be seen as a special case of the t-limited scheme proposed by Stern and 
Dill IE] with t = 1. One disadvantage of this scheme is that entries will already 
get overwritten before the table is completely full. Since for the “interesting” 
(difficult) cases, the state table will fill up soon anyway, this effect is limited. 



5.5 Backward Search 

Many puzzles are symmetric^ i. e., the set of children of a state equals the set of 
possible parents. This is equivalent to the state space graph being undirected. 
As already mentioned, this is the case for the 15-puzzle, but not for Sokoban 
or Atomix. For Atomix, it is simple to find all potential parent states, though: 
they can be found by applying all legal backward moves. In a backward move, an 
atom being pushed may stop moving at any position, but it can only be pushed 
in a direction if it is adjacent to an obstacle in the opposite direction. 

Formally defined, a backward move is a triple of a position p, a direction d, 
and a distance S. It is legal for a state s if there is an atom (a,p) in s, and 
{px—dx,Py—dy) is not empty, and (px+S'dx,Py+S'dy) is empty for all 0 < < <5. 

Applying a backward move is analogous to applying a forward move. 

Expanding states for backward Atomix is about as easy as for forward 
Atomix, and the same heuristic can be used, since the generalized moves from 
Sect. 15.11 comprise backward moves. Hence, the crucial point is the branching 
factor. 

Lemma 1. The sum of possible forward moves and the sum of possible backward 
moves of all states of a level are identieal and, therefore, the average number of 
children for backwards expansion is exaetly the same as for forward expansion. 

Proof. We first show the equality for a single atom by structural induction. On 
a board with no empty squares, the equation is trivially true. We show it also 
remains true when removing an obstacle. The change in the number of moves 
depends on the pattern of empty squares around the obstacle being removed; 
we examine all possible patterns (up to symmetry, and omitting the trivial case 
of 4 obstacles), as illustrated in Fig. El with a,b,c and d being the number of 
empty squares in each direction. 

(a) 3 adjacent obstacles: 1 — 6-l-6-l-l = l-l-l = 2. 

(b) 2 adjacent obstacles, where the obstacles are diagonally adjacent: 
l-b+b + d+2-d+l = l + 2+l = 4. 

(c) 2 adjacent obstacles, where the obstacles are opposite: 
c-t“2 — 6-t“0 — c b 2 = l-t-2-t-l =4. 

(d) 1 adjacent obstacle: c-|-2 — 6-|-d-|-l — c-|-6-l-2 — d-|-l = l-l-3-|-l-l-l = 6. 
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Fig. 5. The light grey obstacle in the center is being removed. The upper left corner 
of each square denotes the number of backward moves that are lost or gained by this 
change for an atom on this square. The lower right corner denotes the number of new 
forward moves. Squares which are skipped in the sketches (denoted by dots) have zero 
gain with respect to both forward and backward moves. 



(e) no adjacent obstacles: 

d -\- ‘2 — d -\- c -\- 2 — 6-t“0 — — d-t-n“t“2=l-t-l“t“4-t-l“t“l = 8. 



Now, let us consider the contribution of one atom to the possible moves. 
Each possible distribution of the other atoms can be considered as a pattern 
of obstacles. With the observation just made, the sum of possible forward and 
backward moves is the same when summing up over all possible positions of the 
considered atom; so the sum over all possible distributions of the other atoms is 
also identical and, since this equality holds for each atom, the lemma is true. □ 
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In practice, the branching factors can differ substantially, since the generated 
states are not random; the move operators make certain states more likely than 
others, and states close to the goal where (by convention) all atoms are close 
together are much more likely. In our experiments, we observed differences up 
to 30% in forward and backward branching factors. 



6 Implementation 

6.1 Identical Atoms 

The presence of undistinguishable atoms (i. e., atoms with identical atom types) 
poses problems for an implementation: The heuristic cannot simply perform a 
table lookup to find a lower bound for an atom, since it is not clear which 
atom should go to which goal position. To find a good lower bound, a minimum 
cost perfect matching has to be done for each set of identical atoms to find the 
cheapest assignment of atoms to goal positions. Minimum cost perfect matching 
for a bipartite graph can be solved using minimum cost augmentation in time 
quadratic in the number of identical atoms M- 



6.2 A* 

An implementation of A* needs the following operations: check if a state has been 
encountered before and with which g-value, find an open state with optimal /- 
value, mark an open state as closed, and update the g-value of a saved state to 
a lower value. 

This is usually implemented with a hash table and a priority queue which 
stores all open states. We will show that if the heuristic is monotone, no priority 
queue is actually needed: an optimal open state can be found efficiently without 
any additional data structures. Our algorithm is easy to implement and time 
and space efficient. 

Initially, the available memory is allocated for two tables: the state table and 
the hash table. As states are generated, they are appended to the end of the state 
table; states never get deleted. The states are tagged with an open-hit and with 
the g-value. The hash table stores a pointer into the state table at the position 
corresponding to the hash value of the state; this allows a quick lookup of states. 
A linear displacement scheme is used to resolve hash collisions. The monotonicity 
of the heuristic implies that /opt, the currently optimal /-value of an open state, 
is also monotone over the run of A*. To find an optimal open state, a linear 
search on the state table is performed until an open state with / = /opt is found. 
The following proposition shows that this can be done efficiently: 

Proposition 3. In A* with a monotone heuristic with a hash table and no 
additional data structure, a state with optimal f -value ean be found in amortized 
time 0(branching factor). 
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Proof. To achieve this, we need to ensure that, for each /opt-value, when we 
reach the end of the state table, we have expanded all states with / = /opt> 
so we don’t have to go through the table again. This can be ensured by not 
upgrading a state in place if it is re-encountered with lower g, but to append it 
at the end like new states. States with / < /opt will never be reopenened jS|, so 
this suffices to ensure the desired property. 

Two kinds of states will be skipped because their /-value differs from /opt: 

— Closed states with / < /opt. We keep a pointer to the very first open state, 
so only closed states with / = /opt — 1 or / = /opt — 2 have to be skipped; 
for any branching factor greater than 1, this can be at most twice as many 
as states with / = /opt and, with a higher branching factor, their number 
even becomes negligible. 

— Open states with / > /opt. They must have been generated by states with 

/ = /opt or / = /opt — 1, so their number is linear in the number of states 
with / = /opt and the branching factor. □ 

Our implementation with this scheme is several times faster than a naive 
implementation using the C-|— I- STL priority_queue and set, which are based 
on heaps, resp., binary trees, with a memory overhead of about 30 bytes per 
state. On a Pentium III with 500 MHz, it can generate around a million states 
per second. 

A disadvantage of this scheme is that it is not possible to further discriminate 
among optimal states. A common idea to speed up A* is to sort among states 
with equal /-values those closer to the top that are further advanced. 

To trade time for memory, the A* implementation works iteratively: similarly 
to IDA*, an artificial upper bound on the number of moves is applied and, if the 
/-value of a generated state exceeds this bound, it is pruned. If then the search 
fails, it is restarted with the bound increased by one. This also allows us to take 
multiple goal positions into account. Due to the exponential behavior, this slows 
down the search only by a constant factor. 

7 Conclusions 

Atomix proved itself to be a challenging puzzle; this is corroborated by the re- 
cent PSPACE-completeness proof. The classic algorithms A* and IDA* have 
been implemented and adapted to the problem domain; we have found optimal 
solutions for many problems from our benchmark set. Our A* implementation 
with a single data structure for the open and closed set can solve “smaller” 
puzzles very efficiently. With Partial IDA* based on hash compaction, we have 
presented a memory-bounded scheme that makes excellent use of the available 
memory and has low runtime overhead; improved bounds on the error probabil- 
ity would be useful, though. Further progress is likely to come from improved 
heuristics rather than from better search methods, since our current heuristic is 
rather uninformed. We have shown that while the search graph is directed, the 
backward branching factor does not differ from the forward branching factor; 
this makes Atomix an interesting testbed for bidirectional algorithms. 
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A Experimental Results 

The experiments were performed on a Pentium III with 500 MHz, utilizing 
128 MB of main memory and imposing a time limit of one hour. The source 
can be found at http://www-fs.informatik.uni-tuebingen.de/~hueffner. 
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Time performance. A* runs out of memory usually much before a runtime of 
one hour and, so, can establish less stringent bounds. The advantage of using 
a transposition table for IDA* outweighs its runtime overhead and yields bet- 
ter results in all cases. Reverse search performs similar to forward search, as 
founded by the theroretical findings. Partial IDA* consistently beats IDA* with 
conventional hash tables because of better memory utilization and less runtime 
overhead. Note that most of these differences are expected to be more significant 
if the time limit is increased. 
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Abstract. When agents like mobile robots discover that the world is not as ex- 
pected after carrying out a sequence of actions, they are interested in what action 
failures or unnoticed actions could have actually occurred, which would help them 
rectify the situation. For this purpose, we investigate a kind of history-based diag- 
nosis which is appropriate for explaining what went wrong in dynamic domains. 
It turns out that there are often many diagnoses which are quite similar and differ 
only in the objects they refer to. In this paper we show how these instances can be 
compactly represented by introducing so-called diagnosis templates. We formal- 
ize this approach for an action theory based on the situation calculus and discuss 
a prototypical implementation of a diagnostic system which generates diagnosis 
templates according to certain preference criteria. 



1 Introduction 

Agents who carry out a course of actions inevitably run into the problem that things do 
not work out as planned. For example, a robot delivering a book may end up losing the 
book along the way or delivering it to the wrong room. Finding out what went wrong 
and recovering from it is a difficult problem. In contrast to much traditional work on 
diagnosis where the focus is on the analysis of “what is wrong”, diagnosis in settings 
like mobile robots acting in a changing environment focuses on “what happened” which 
we refer to as history-based diagnosis. 

Given a description of system behavior and, potentially, an (assumed) history of 
occurred events, the diagnostic task arises from a contradicting observation. Diagnoses 
that explain observations by conjecturing what happened since an initial situation (i. e., 
without an explicit history) are, e. g., presented in lllOldll . In 111 1 1 so-called explanatory 
diagnoses are studied which are continuations of a given history. It is shown that this 
kind of diagnosing is analogous to planning. 

In our approach to diagnosis we allow adding events not only at the end but at 
any point of the history. In addition to that we exploit another source of explanation 
by taking into account the possibility that some history events/actions might not have 
happened as assumed (or might not have occurred at all). Obviously, in environments 
with uncertain knowledge about occurrence and outcome of events this kind of reasoning 
is very important, as is the former one. So both have to be combined, yielding history- 
based diagnoses. 
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It turns out that there are often many diagnoses which are quite similar and differ only 
in the objects they refer to. In this paper we show how these instances can be compactly 
represented by what we call history-based diagnosis templates. Moreover, sometimes 
there are explanations of the observation that are representable by diagnosis templates 
but not by diagnoses (cf. Theorem IT?] - Remark fTO . Furthermore, diagnosis templates 
can be effectively computed. 

History-based diagnosis can be used as (and is intended to be) part of an execution 
control system of an autonomous agent. In Q a situation-calculus-based execution 
monitor is presented which assumes “that all discrepancies between the robot’s mental 
world and reality are the result of exogenous actions, and moreover, that the robot 
observes all such actions” or at least the effects of these actions. Although this approach 
is more general than it hrst appears it still does not take account of the information that 
can be gained from doubting the (assumed) history. For example, when delivering a book 
a robot mostly is not able to recover from the sole observation that the book is no longer 
where it should be whereas the information that the book was lost while carrying it from 
one room, Rl, to another room, R2, or that it was delivered to the wrong room, R3, may 
help a lot. History-based diagnosis also addresses in an explicit manner the problem that 
there may be a delay between the occurrence of an action failure or of an unexpected 
event and the observation of its possibly indirect effect(s). This is not done in 0] nor in 
implemented systems like SPEEDY 0 and Rogue (6J. 

The rest of the paper is organized as follows: In the next section we present an example 
application which we refer to repeatedly. In the following two sections we address some 
aspects of the situation calculus used in this paper and present the formalization of 
our approach to history-based diagnosis. Then the subject of diagnosis preferredness is 
discussed. Afterwards we introduce diagnosis templates. In the following two sections 
we outline how most preferable diagnosis templates can be computed. In the final section 
we give a brief summary and outlook. 



2 A Robot Example 

As an example, let us consider an autonomous robot whose task it is to bring book B 
from room Rl into room R2. Suppose the robot is in room Rl already. The robot decides 
(plans) to carry out the sequence of actions 

fj* = [pickup{B), startfor{R2), arriveat{R2),putdown{B)] 

and initiates its executionH In the situation obtained after the (assumed) execution of 
the four actions, fj* is the (assumed) history and it is derivable that B ought to be in R2. 
Now the robot receives the message (e. g., by the disappointed would-be recipient) that 
B is not in R2. This contradicts the assumed history fj* . But what happened actually? 
Some explanations are: 

(1) The robot lost B on its way to R2. 

(2) The robot lost its way and entered room R3 instead of R2. 



* Assume a navigation software as in Q which usually, but not always leads to successful 
navigation from one room to another. 
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(3) The robot failed to grip B during the pickup-action. 

(4) Somebody took away B after the robot had put it down in R2. 

In case (3) a “failure variation” of pickup, say pickup', happened instead of the “real” 
pickup -action. 

The four explanations correspond to four diagnoses which are modified histories 
explaining the fact that B is not in R2: 

5® = [pickup{B), startfor{R2), robotloses{B), arriveat (R2) , putdown'] 

= [pickup{B), startfor{R2), arriveat(R3), putdown{B)] 

= [pickup' , startfor (R2), arriveat(R2),putdown'] 

= [pickup{B), startfor(R2), arriveat(R2) , putdown(B) , somebody takes (B)] 

In cases (1) and (3) the “real” puf down-action could not have taken place since it is 
necessary to have an object in order to put it down. Therefore d® and d® contain a 
putdown' -action instead of the putdown-action. (Note that only d® is a continuation of 

r-) 

Of course, there are many other explanations resp. diagnoses. However, there are 
explanations that should not be considered a valid diagnosis, e. g. 

a = [pickup{B), startfor (R2), arriv eat {R2) , putdown{B) , 
pickup{B), startfor[Rl), arriveat{Rl) , putdown{B)] 

The reason why a is not a diagnosis (although it is an explanation of the observation) 
is that the robot would have known if it had brought B back into R1 after bringing it 
into R2 and therefore it had assumed a to be the history instead of fj* . The robot would 
have known if it has executed (or at least initiated) an action like pickup because such 
deliberate actions happen under the control of the robot unlike (exogenous) events like 
losing a book. 

From this simple scenario we can already infer the following requirements: a diag- 
nosis should 

- form a possible history (according to the given description of the system behavior); 

- explain the observation; 

- take into consideration the history assumed so far, i. e., include (in the corresponding 
order) all history events/actions or variations of them; 

This is due to the assumption that an event/action occurs in the history because 
the agent has good reason to presume something has happened (e. g., there may be 
uncertainty about the actual effects of actions, but not about their initiation)^ 

In the robot example, pickup' is a variation of pickup (B), and arriveat{R3) is a 
variation of arriv eat (R2) . 

- use as additional events only suitable "insertions ”, i. e., such events that are not under 
the agent’s control but may have occurred and can help to explain the observation. 
In the robot example, robotloses{B) and somebodytakes{B) are such insertions. 

Note that in the situation calculus there is no formal distinction between actions and 
events: both are called actions (and formalized equally). 

^ In principle, the non-occurrence of an event/action can be represented by a special dummy 
event/action as a variation, e. g., by an event/action without any effects. 
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3 Situation Calculus 

To formalize our approach we use the situation-as-histories variant [f^l of the situation 
calculus |2!] and assume the reader to be somewhat familiar with it. Nevertheless most of 
the material presented here can be understood intuitively. In this section we only mention 
some aspects of the general situation calculus but describe extensions that we use in 
the context of history-based diagnosis. We adopt the convention that free variables are 
implicitly universally quantified unless otherwise stated. <P\xi, . . . , Xn \ indicates that 
the free variables of the formula <P are among xi, ... ,Xn- 

A basic action theory T> describes the initial state of the world and how the world 
evolves under the effects of actions. Among others, it consists of successor state axioms 
Ea and action precondition axioms. For instance, if A is a n-ary action function, an 
action precondition axiom of the form 

Poss(A(a;i,...,a;„),s) = TTa [xi, . . . , x„, s] 

is used to state under which condition, II a, it is possible to execute action A(x\ ,... , x„) 
in situation s, e. g., 

Poss{putdown{x), s) = Having{x, s) A -iMoving(s) 

Here, in the context of history-based diagnosis, the basic action theory T> additionally 
contains 

- an action variation axiom for each action function 

- an insertion axiom 

which use predicate symbols Varia and Inser that are conceptually similar to Pass. 
Action variation axioms of the form 

Varia{a, A{xi, . . .,Xn),s) = &A\a,xi, . . . ,x„, s] 

are used to state under which condition, 0a, an action, a, is a valid variation of another 
action, A(xi, . . . , x„), in a situation, s, e. g., 

Varia{a, putdown(x) , s) = a = putdown' V 3y [a = putdown{y) A Having{y, s)] 
Varia{a, arriveat(r) , s) = [a = arriveat(r') A Room{r')] 

Mostly, as in the examples given above, Poss{a, s) implies Varia{a, a, s), i. e., an action 
is a variation of itself if it is possible to execute it0 
An insertion axiom of the form 

Inser{a, s) = 0[a, s] 

states under which condition, 0, an action, a, is a valid insertion in a situation, s, e. g., 

Inser{a, s) = 3z[a = robotloses{z) A Having{z, s)] 

V [a = somebody take s{z) A Portable{z)] 

Insertions are suitable additional actions that may have occurred and can assist in diag- 
nosing. Typically, insertions are actions (events) that are not under the agent’s control. 

^ Sometimes there are good reasons for variation relations without this property. But we do not 
consider such applications here. 
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To simplify matters, we assume that the variations and insertions are disjoint, i. e., 
T> 1= -i3a,s [3a' Varia{a, a' , s) A Inser{a, s)]. 

There is a syntactic restriction on the formulas II a • , s] , 0a • j s] , (9 [. . . , s] : 

they are uniform in s, i. e., they are formulas such that, if they contain a situation term 
at all, then that term is s and there is no quantification over s. 

In what follows we also need the notion of situation-suppressed formulas, i. e., for- 
mulas where all occurrences of situation terms are “deleted” (details omitted). If </) is a 
situation-suppressed formula then (p [cr] denotes the situation calculus formula obtained 
after restoring suppressed situation arguments by “inserting” the situation cr where nec- 
essary, e. g., (p = ^Moving A 3a; (Book{x) A Location{x, R)) is a situation-suppressed 
formula and </)[cr] = -tMoving{a) A 3a; (Book{x) A Location{x, R, a)). 

We denote action sequences in square brackets, like [cri, . . . , a„]. do{a, a) denotes 
the situation obtained after the execution of action sequence a in situation cr0 The 
constant Sq denotes the initial situation. We abbreviate (p[do{a, 5o)] by (p\a\ for every 
situation-suppressed formula (p and use the abbreviation 

Exec{[ai, a„]) = AjG{i.....n} Poss{aj, do{[ai, ^o)) 

Exec{[ai , . . . , «„]) expresses that starting in the initial situation Sq it is possible to 
execute the actions a±, ... ,an one after another. A ground action sequence a is called 
executable iff T> |= Exec{a). Finally, a.a = [«i, . . . , a] when a = [«i, . . . , a„]. 

4 History-Based Diagnoses 

A history is simply a ground action sequence fj. An observation is a situation-suppressed 
closed formula p. The diagnostic task arises if the observation contradicts the assumed 
history, i. e., T> \= . The (assumed) history and observation of the robot example 

are 

fj* = [pickup{B), startfor{R2), arriveat{R2),putdown{B)] 
p* = ->Location(B, R2) 

Here V ^ Location{B, R2, do{fj* , Sq)), therefore V \= whereas V \= p’‘[fj] 

should hold for the “real” history fj. 

Next we address the rather syntactic qualities of history-based diagnoses. 

Definition 1. An extended variation of a ground action sequence a = [«i , . . . , cr„] is a 
ground action sequence 6 = [(5i , . . . , 6m] such that 

- a mapping t : {1, . . . , n} — >■ {1, . . . , m} exists with (,(1) < • • • < t(n) 

- for each i G {r(l), • ■ • , t(n)} with i = c{j): 

6i is a valid variation of aj in the situation after the action sequence [5i, . . . , A-i], 
i.e.,V \= Varia{5i,aj, do{[5im . . ,6i_i], Sq)) 

- for each i G {1, . . . , m} \ {t(l), . . . , t(a)}: 

5i is a valid insertion in the situation after the action sequence [i5i, . . . , A-i], 
i. e., V \= Inser{Si, do{\6i , . . . , 5o)) 



4 



do([ai, . . . , an\,o) abbreviates do{an, . . . do{ai,a) . . .). 
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The property of S being an extended variation of a can be formulated as a situation cal- 
culus formula (Lemma|2I). For this purpose we introduce an abbreviation Ext Vari ( a) 
by 



ExtVari{[], []) 
ExtVari{[], a.a) 
ExtVari{S.S, []) 
ExtVari{5.6, a.a) 



True 

Ealse 

Inser{5, do(S,So)) A ExtVari{6, []) 

[Varia{5,a, do{5,So)) A ExtVari{5^a)] 
V [Inser{5, do{S, Sq)) A ExtVari{5, a.a)] 



The first and second equation together mean that [ ] is an extended variation of no other 
action sequence but []. The third equation means that 6.S is an extended variation of [] 
iff ^ is a valid insertion in the situation do{6, Sq) and 6 is an extended variation of [] 
itself. The fourth equation means that 6.S is an extended variation of a.a iff {!.) 5 is a 
valid variation of a in the situation do(j, Sq) and S is an extended variation of a or (2.) 
5 is a valid insertion in the situation do(S, Sq) and S is an extended variation of a.a. 



Lemma 2. A ground action sequence d is an extended variation of a ground action 
sequence a iff V |= ExtVari{6, a). • 

Now it is easy to define history-based diagnoses. 



Definition 3. An explanatory history-based diagnosis for an observation (f) and a 
history fj is an extended variation 5 of fj such that 6 is executable and (p holds in the 
situation after the action sequence 6, i. e.,T> \= (p [do{6, ^o)] . ■ 

Definition 0 captures all of the above-mentioned requirements and can also be for- 
mulated as a situation calculus formula (Lemma|3|). We introduce the abbreviation 

ExplDiag{6,<p,fj) = ExtVari{6,fj) A Exec{6) A (p[6] 



Lemma 4. A ground action sequence 6 is an explanatory history-based diagnosis for 
an observation p and a history fj iff I) \= ExplDiag{6, p,fj). ■ 

In the robot example , . . . , 5® are explanatory history-based diagnosis for obser- 
vation p* and history fj* . Other simple diagnoses are for instance 

= [pickup{A), startfor{R2), arriveat{R2),putdown{A)] 

= [pickup{B), startfor{R2), robotloses(B), arriveat{R3) , putdown'] 

5® = [pickup{A), startfor{R2), arriveat{R3),putdown'] 

but 5® should not be considered a preferred diagnosis since it combines 5® and 5® and 
hence “over-explains” p* in a way. The same is true for 5® . 
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5 Preference Values 

In most cases there are many diagnoses for a given observation and history (in fact, 
infinitely many). Therefore criteria are needed to decide which diagnoses are preferred. 
The number of modifications between a diagnosis and the history, i. e., the number of 
insertions and proper variations in a diagnosis, yields a simple preference criterion: the 
higher that number is the less preferred a diagnosis is. When comparing 

5*-®^ = [pickup{B), startfor{R2), arriveat{R2), putdown'] 

6^^ = [pickup{B), startfor(R2), arriveat{R3),putdown(B)] 

= [pickup' , startf or (R2), arriveat (R2), putdown'] 

a problem with this number-of-modifications criterion can be seen: There is one modi- 
fication in ; in there are two, but the putdown-f sd\uK is merely an aftereffect of 

the pickup -failure and hence should not be counted. 

A somewhat more sophisticated preference criterion (which seems to avoid this prob- 
lem) is comparing diagnoses by inspecting the modifications (not only counting them): 
a diagnosis is more preferred than another diagnosis if its modifications are a proper 
subset of the modifications of the other diagnosis. However, this set-of-modifications 
criterion does not allow to compare arbitrary diagnoses but only those whose modi- 
fication sets are ordered by the subset relation. Furthermore, according to this set-of- 
modifications criterion is more preferred than But it may be the case that an 
initial pickup-failure (which entails a putdown-failure) is more likely than an isolated 
putdown-f allure and therefore 5^®^ should be more preferred than 5®. 

So we look for preference criteria that can subsume the set-of-modifications criterion 
but also can regard entailed actions and allow to compare all diagnoses with a more subtle 
ranking than the number-of-modifications criterion. For this purpose we use preference 
valuations which are functions that assign a preference value p^_^(5) > 0 to every 
extended variation 6 of the history fj (hence to every diagnosis, too) such that higher 
values for diagnoses correspond to more preferred diagnoses^Particularly we exploit the 
special structure of normalized inert modular preference valuations which are defined 
at first and explained after that. The idea behind modularity is that the preference value 
of a diagnosis can be determined using kind of preference values for the single diagnosis 
actions. 

Definition 5. p^^^ is normalized iff = 1. ■ 

Definition 6. p^_^ is modular iff for each pair {S, S') where S'.S is a first part of an 
extended variation of fj there is a value p0,^(5 1 S') such that 0 < prj,^fj{S | ^0 ^ 

P<f>,f)(S) = p(fi,fjiv) ■ riie{l,. ..,»«} I ■ • ■ ’ (*) 

for every extended variation S = , . . . , <5^] of V- ■ 



^ Note that, in the majority of cases, for a given preference criterion one can find a corresponding 
preference valuation subsuming the preference criterion. 
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Corollary 7 . If is modular then | [771, , 77^]) = 1 . 

Definition 8. is inert modular iff is modular and (77^+1 1 5 ') = 1 when 
77j+i is the next history action that is not “covered” by 5 ' (i. e.: 771 , . . . , r]j are the history 
actions for which variations are in 5 ')- ■ 

Theorem 9 . If a preference valuation respects the set-of-modifications criterion then it 
is inert modular. • 

Let us now have a look at the role of the factors p<j,,fj {6 \ S') in Equation ( 01 . Since 
0 < P<i>,f){S\S') < 1 this factor specifies how the modification associated with 5 (i. e., 
the replacing of a history action by 5 or the inserting of 5 ) diminishes the preference 
value of the obtained action sequence w. r. t. the preference value p<j>,fj{f]) of the history. 
The intuitive meaning of p^^fj{Si \ [i 5 i, ■ . ■ , is: 

- If Si is a variation of the next history action 77^+1 that is not “covered” by < 5 i, . . . , 

Si-i (i. e.: 771, . . . , rjj are the history actions for which variations are among Ji, . . . , 
( 5 i_i) then p 4 ,^fi{ 5 i \ [< 5 i, . . . , is the diminishing factor corresponding to next 

replacing 77^+1 by Si when the first part of a diagnosis is [( 5 i, . . . , Si-i]. 

If p0,ij is inert then p4>,f){S^ | [^1, . . . , = 1 when Si = 77^+1, i. e.: 

not-replacing a history action does not diminish the preference value. 

- If Si is an insertion then p(f,^fj{Si | [< 5 i, . . . , is the diminishing factor corre- 

sponding to next inserting Si when the first part of a diagnosis is [Ji, . . . , 

In both cases we may have p^^f){Si | [ 5 i, . . . , < 5 i-i]) = 1 if [i 5 i, . . . , Si-i] entails Si. 

If p^^fj is modular then 0 < p^^fj{S) < p^^fj(fj) for every extended variation S of the 
assumed history 77, i. e., fj has the maximal preference value. This is intuitively right: 
Assuming that the observation does not contradict the history then the history itself is 
a diagnosis and, of course, the most preferred one. p^^fjid) should not equal 0 because 
otherwise all preference values are 0 . If p^,fj{v) ^ 0 then p^ ^^ can be normalized. If 
p^^f) is normalized then 0 < p^^fj{S) < 1 and 

■ ■ ■ 1 Sm]) = nig{l,...,m} P 4 >,r){Si | [( 5 i , . . . , 

So p0,7j(5) and p^^fj{Si \ [< 5 i, . . . , have the flavor of (conditional) probabilities. 
But they are no probabilities! For instance, J 2 s P<f>,v(S) = 1 does not hold in general. If 
every P<f>,v(S I S') is finite one can transform the preference values to probabilities by 
normalizing them. But then p^^f](jj) < 1 - In Cl an approach is shown how probabilities 
can be utilized as preference criterion@ 

Without the requirements of probability distributions we are free to assign intuitive 
values between 0 and 1 (0 and 1 included) to p0,7;(i5| S') with the only restriction that 
pcji^rjiv I ^0 = 1 must hold for the next history action 77 (see above). In doing so we get 
an inert modular preference valuation p^ by means of Equation ®Q The preference 

® The “features” that are used in □ to reduce the complexity of determining the probabilities 
are comparable with the value-conditions x that are used later in this paper. 

The number-of-modifications criterion (which, of course, subsumes the set-of-modifications 
criterion) can be emulated by assigning a fixed value other than 0 or 1 to all p<i,,fj{S \ S') that do 
not have to be 1. 
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values provide a preference criterion that can subsume the set-of-modifications criterion 
but also can regard entailed actions and allows to compare all diagnoses with a possibly 
more subtle ranking than the number-of-modihcations criterion. Note that we only have 
to provide values for ((5 1 S') ifj is a valid variation or insertion in situation do{S' , Sq). 



6 Diagnosis Templates 

Recall the history and observation of the robot example: 

fj‘ = [pickup{B), startfor(R2), arriveat(R2),putdown(B)] 

(jf = -> Location {B, R2) 

For instance, there may be many diagnoses that arise from erroneously picking up the 
wrong book in the beginning: 

[pickup{C), startfor{R2), arriveat(R2) , putdown(C)] 

[pickup{D), startfor{R2), arriveat{R2) , putdown{D)] 

[pickup{E), startfor{R2), arriveat{R2) , putdown{E)] 

They all are instances of the non-ground action sequence 

S = [pickup{x), startfor(R2), arriveat(R2),putdown{x)] 

But not all instances of 6 are diagnoses, e. g., with {x i— B}, most of them are not 
even executable or extended variations of the history, e. g., with {x i— Rl}. In order to 
further restrict the (ground) instances of (possibly non-ground) action sequences we use 
parameter constraints. 

Definition 10. A parameter constraint for an action sequence 5 is a formula 'L such 
that the free variables of W are exactly the variables in d. • 

We call S' an instance of (6, ff) iff an assignment v of the variables in 5 to terms 
exists such that V ^ 3'Lu and S' = Sv. If Sv is ground then V ^ L'v must hold. 

Definition 11. An explanatory history-based diagnosis template for an observation (j) 
and a history ^ is a pair {5, L') consisting of an action sequence S and a parameter 
constraint for S such that 

V ^ V[<?' D ExtVari{S,ff)] A A Exec{S) A 

(SjE) is called universal iff 22 ^ \/[<E A Exec{5) D 

(S, E) is called executable iff 22 |= \/[E D Exec(S)]. ■ 

For example, if the history and observation are 

p = [pickup (A) , pickup (B) , startfor(R2) , arriveat(R2)] 

(j) = ->Location{B, R2) A 3x Having{x) 

then with a = do{[pickup{A) , pickup{B) , startfor (R2)], So) both {S', E') where 




History-Based Diagnosis Templates in the Framework 253 



S' = [pickup{A) ,pickup{B) ,startf or (R2) ,robotloses{x) ,robotloses{y) ,arriveat{R2)] 
W' = Having {x, a) A Having{y, do{[rohotloses{x)],u)) 
and (5", S'") where 

S" = [pickup{A) ,pickup{B) ,startf or {R2) ,rohotloses{x) ,arriveat{R2) ^rohotloses{y)] 
— Having{x,a) A Having{y, do{[robotloses{x), arriveat(R2)],a)) 

are executable diagnosis templates0The first one is universal and all its ground instances, 
one with {a; i-A A,y i— > B} and one with {ati-AB, 2 /i-^A}, are diagnoses. The second 
one is not universal since, because of cj), its instance with {x i— >■ A, t/ 1 — > B} is not a 
diagnosis while its instance with {a; i— B, y A} is. 

From Definition im (resp. Corollary 113 one can see: (i.) all ground instances of 
a diagnosis template are extended variations of the history; (2.) if it is universal then 
all executable ground instances are diagnoses; (3.) if it is universal and executable then 
all its ground instances are diagnoses. But it is not necessarily true that at least one 
ground instance exists that is a diagnosis (see below). This is an advantageous feature 
of diagnosis templates (cf. Remark^^. 

Corollary 12. Let (S, <A) be an explanatory history-based diagnosis template for an 
observation (jj and a history fj. Then T> \=3['I' A ExplDiag{S, if), fj)]. 

If {S, E) is universal then T> |= V[!A A Exec{S) D ExplDiag(S , 4>, fj)]. 

If {6, E) is universal and executable then T> ^ V[!A D ExplDiag{5, fj)]. • 

To every diagnosis template (S, E) the set ^(5, E) of all its ground instances that 
are diagnoses is assigned. Let be the union of all the !?'). Note that for 

every diagnosis 5 (S, True) is a diagnosis template with True) = {5}. Hence: 

Theorem 13. is the set of all diagnoses for observation (j) and history fj, 
i. e., we do not miss any diagnosis when considering diagnosis templates only. 

We do not only not miss any diagnosis but sometimes we actually gain evidence for 
“diagnostic explanations” that are not expressible by a diagnosis but are representable 
by a diagnosis template. For instance, assume that in the robot example the robot has 
the information that in the beginning there are books in room R1 beside B but no further 
information is given about these books (particularly, the robot does not know which 
books). Then (S, T) where 

S = [pickup{x), startfor{R2), arriveat{R2), putdown{x)] 

= Book{x) Ax ^ B A Location{x, Rl, Sq) 

is a diagnosis template but none of its ground instances is a diagnosis. In fact, it is an 
universal and executable diagnosis template such that no ground instance exists at all 
(since no ground term t exists such that T> \= E{x i— >■ t } ). 

Remark 14. By means of diagnosis templates it is possible to find “diagnoses” (meant 
in an informal way) that cannot be expressed as diagnoses (as defined by Definition|3- ■ 



Suppose that in So the robot does not carry anything and is located in the same room as the 
books A and B. 
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Of course, {S, ExplDiag{6, </>, fj)) is an executable universal diagnosis template iff 
T> 1= 3ExplDiag{5, fj). The problem with this diagnosis template is that nevertheless 
one has to compute the action sequence S, thereby ensuring T> |= 3ExplDiag{S, (j), fj). 
In the next section we will show how we can determine simpler parameter constraints 
S' for S than ExplDiag{S, 4>, fj) along with the computation of 5. 

When computing diagnoses and diagnosis templates we would like the computa- 
tion to be guided by preference values. In general there may be instances of {5, W) 
that have different preference values. So, what is the preference value of {5, E)! The 
minimum? The maximum? Our approach here is to consider only preference-uniform 
diagnosis templates, i. e., diagnosis templates (6, 'E) such that there exists a value v with 
Prj>,fj{6') = V for all ground instances 6' of {S, E). This value v is the preference value 
of the preference-uniform diagnosis template (5, E). Note that we do not miss any di- 
agnosis when we restrict ourself to preference-uniform diagnosis templates: For every 
diagnosis 5 {S, True) is a preference-uniform diagnosis template. 

7 Appropriate Basic Action Theories 

In this section we use a situation calculus function symbol pref denoting the preference 
valuation Especially, p^^fj{S \ 5') is denoted by pref {5, a) with a = do{6' , Sq). It 
is not necessary (but possible) to really formalize pref within 2? since all the places 
where we use pref in this paper are only intended to clarify our concepts0 

Given an observation (p and a history fj, our aim now is to compute preference-uniform 
explanatory history-based diagnosis templates (S, E) and their preference values. For 
this purpose we use action variation axioms of the form 

Varia{a, A{xi, . . . ,Xn),s) = 0^^^ \a,Xi , . . . s] 

V •• • 

V \a,xi,. . . ,Xn,s] 

Each of the [a, a:i, . . . , s] has the form 

3yi,...,ym[a = A O^f'lyi,. . .,ym,xi , . . . 

where is a (possibly non-ground) action, yi, . . . ,ym are the variables occurring in 
, and is a formula which is uniform in s. Note that 

T> ^ ^ Varia{5^^ , A{xi , . . . ,x„), s)] 

That is, 0^^ states a condition under which 5^^ is a valid variation of A(xi, . . . , x„) 
in situation s. The action variation axioms given in the situation calculus section are 
examples for that already, e. g.: 

Varia{a,putdown{x), s) = a = putdown' 

V 3y [a = putdown{y) A Having {y, s)] 

® After formalizing the range of , if f is a term for \ S') and a = do{5' , So) we will 

have T> |= pref {5, a) = t. 
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where = putdown' , 6»J,tLo™n = True 

^putdown = putdown {y) , = Having {y, s) . 

In the next section these 9\ are used to build up the parameter constraints of diagnosis 
templates. 

(i) 

Furthermore we provide the algorithm with finite non-empty sets X\’ of pairs 
, t/mi . . . , s] , w) where 0 < w < 1 and x is a formula which is uni- 
form in s. The intuition behind this is that x is a condition under which the preference 
value is w, i. e., 

V A X D pref{6^j\s) = w] 

Actually, the preference values for variations S are defined by the values 

given in the In the robot example (where A(x) = putdown{x)) we may have 

^pltdown = {{-^Having{x, s),l) , {Having{x,s),w^^'>)} 

^iutdown = {(y = Having (x,s) A y x, 1), {Having{x,s) A y 

(y = X, 1) is necessary to ensure the preference valuation to be inert. The other both I’s 
are chosen because the associated x’s are conditions for variations that are aftereffects: 
if the robot is not having x then putdown{x) cannot be executed. Hence, for instance, 
when variations of putdown(x) are concerned 

T> ^ \/[->Having{x, s) D pref {putdown' , s) = 1] 

T> ^ V[ifomny(y, s) A Having{x, s) Ay ^ x D pref {putdown {y) , s) = 

To avoid contradiction it is necessary for (x, w) and (x^ w') in with w ^ w' 
that X and exclude each other w. r. t. 9^^ , i. e., T> ^ D -■(x A x')]- Iii order to 
provide a preference value for every diagnosis and diagnosis template we must require 

No wonder that we treat the insertion axiom similarly, i. e., we use an insertion axiom 
of the form 

Inser{a, s) = [a, s] V • • • V s~\ 

Each of the 0^^'>\a,s\ has the form 

3yi,...,ym [a = 6‘'^^ A [yi, . . . , y^, s]] 

where 6^^'^ is a (possibly non-ground) action, yi, . . . , y^ are the variables occurring in 
and 9^^'> is a formula which is uniform in s. Note that 

V 1= D Inser{S^^\ s)] 

That is, 9^^'^ states a condition under which 6^^'^ is a valid insertion in situation s. The 
insertion axiom given in the situation calculus section is an example for that already: 

Inser{a, s) = 3z[a = robotloses{z) A Having{z, s)] 

V 3z [a = somebody take s{z) A Portable{z)] 
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where = robotloses{z) , 6^^'> = Having{z,s) , 

5 ( 2 ) = somebodytakes{z) , = Portable(z) . 

In the next section these are used to build up the parameter constraints of diagnosis 
templates. 

Furthermore we provide the algorithm with a finite non-empty set of pairs 
{x\yi, ■ ■ ■ ,ym,s],w) where 0 < w < 1 and x is a formula which is uniform in s. The 
intuition behind this is that x is a condition under which the preference value is w, 
i. e., 

T> ^ A X 3 pref{6^^\s) = w] 

Actually, the preference values P4,,f){5\6') for insertions 6 are defined by the values 
given in the To avoid contradiction it is necessary for (x, w) and (x^ w') in 
with w ^ w' that x and exclude each other w. r. t. i. e., T> ^ D “'(x AxO]- 

In order to provide a preference value for every diagnosis and diagnosis template we 
must require D V(x,™)gxw x]- 

Basic action theories whose action variation axioms and insertion axiom are in such a 
way are appropriate for computing preference-uniform diagnosis templates as is shown 
in the next section. Note that with this approach valid variations and insertions are 
independent of the observation. It is a topic under investigation how the given observation 
can be exploited to refine the computation of diagnosis templates. 

8 Computing Diagnosis Templates 

With appropriate basic action theories at hand we are able to define a search space 
in order to compute preference-uniform explanatory history-based diagnosis templates 
{ 5 , If') and their preference values for given observation (j) and history fj = [771, ... , rjn]. 
Each state in the search space is a quadruple (S, 'P,v,j) where 

- 5 is a hrst part of an extended variation of the history 

- If' is a parameter constraint for S 

- u is the preference value assigned with {6, 9) 

- j is the number of the last history action that is “covered” by 5 
(i. e.: In 5 there are variations of the first j history actions.) 

A successor of a state {5, 'P, v,j) is a state (S' , 'P' , v' ,f) such that (with the notations as 
above) 

- = 

If' = ^ So)) 

A xWu ■ • • I do{5, So)) 

v' = V ■ w 

f = J + 1 

where (x,w) G 

y'l, ... ,y'^ are new variables replacing the variables of yielding 
and r]j+i = A{ti, ..., t„), j <n 
or 
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r = rFA9(^Hy[,...,y'^,do_{lSo)) 

/\X{y'l,---,y'm:do{6,So)) 
v' = V ■ w 

f = j 

where {x,w) € 

y{, . . . ,y'^ are new variables replacing the variables of yielding 

Actually, instead of directly assigning ^' = W A 9 A x the assignment is S'' = S' A 0 
if 27 1= y[W A 0 D x] what is checked first. The initial state is ([], True, 1, 0). A state 
{6, T, V, j) is defined to be a goal state iff 27 |= A Exec{5) A 0 [ 5 ] ] and j = nFl 

Lemma 15. If {5, T, v,j) is a goal state then {6, W) is a preference-uniform explanatory 
history-based diagnosis template for (j> and fj with preference value v. • 



Theorem 16. Each diagnosis with preference value v is an instance of some diagnosis 
template (d, T) such that {5, T, v, n) is a goal state. ■ 

There are possibilities to prune the search space: If {5, T, v,j) is a goal state with 
S = [^ 1 , . . . , 5m\ then T — True A A • • • A Tm where each Ti corresponds to Si. 
Since Exee(5) = Poss(5i, CTi) A • • • A Poss(5rmCrm) wither^ = do{[5i, . . . , Sq) 
and 27 ^ A Exee{S)] must hold for a goal state 27 |= A Poss{Si, ai)] are nec- 
essary conditions for being a goal state as well as 27 |= A • • • A Pi A Poss{6i,(Ji)]. 
These conditions can be tested when generating the successors of a search state. This 
analysis also yields that 27 |= 3[P' A Exec{6')] is a (stronger) necessary condition when 
generating the successor {S' , P', v' ,j') of (S, P, v,j). 

The search space given here is well suited for search algorithms that work like 
uniform-cost search except that here the nodes in the search tree that have higher pref- 
erence values have to be expanded first. Note that the branching factor is finite since 
there are only finitely many S[^ and S^^'^ and all of the and are finite. We 
have implemented a prototypical diagnostic system in Prolog that uses an algorithm 
with iterative deepening along the preference values. It also computes for each diagnosis 
template all ground instances that are diagnoses. 

The algorithm is sound and optimal (in the sense that it first outputs diagnosis tem- 
plates with highest preference value). There are weak conditions under which the algo- 
rithm is complete: Assumed that a diagnosis template with non-zero preference value 
exists, it is sufficient for completeness that w < 1 for each (x, w) in each X^^\ This 
restriction is not necessary for the Xj^ . 

We ran several experiment^ in order to get an estimate how computing diagnosis 
templates compares with computing di^noses. For the latter we used an algorithm 
similar to the one outlined here and in El Qln these experiments runtimes were measured 
for computing 

If j = n then 27 |= V[>f" D {ExtVari{5, y) A pref{5) = u)] because of the construction, 
on a Pentium 500 Mhz Linux-PC using ECLiPSe 4.2 

Instead of actions 5 and constraints 9 A x, ground instances of {5, 0 Ax) are used and con- 
straints omitted when computing successors of search states. 
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- all diagnoses with maximal preference value 

- all diagnosis templates with maximal preference value 

and all their ground instances that are diagnoses 

and computing diagnoses directly took 3.65 to 8.33 times more runtime than computing 
them via diagnosis templates. For short histories (like in the robot example so far) the 
system outputs all most preferred diagnoses almost instantaneously (within less than 0. 1 
seconds). In another robot example the history consists of 42 actions and 2.36 minutes 
were needed (compared to 14.5 minutes without using templates). In both robot examples 
the system has to deal with 10 rooms and 1000 books. 

Of course, these experiments only provide a first glance on potential runtime savings. 
We are working on a formal analysis of the complexity and on testing our approach 
with some “benchmark problems” and comparing it with other approaches in order to 
strengthen the experimental results. 

9 Summary and Future Work 

Our concern in this paper has been to formalize how a certain observation can be ex- 
plained by answering the question of what happened instead of an assumed history if 
this assumed history contradicts the observation. We introduced the notion of history- 
based diagnosis in the situation calculus and demonstrated the benefit of this approach 
for diagnosis in dynamic domains. History-based diagnoses are possible histories that 
are extended variations of the assumed history. Furthermore we showed how diagnosis 
preference criteria can be described using preference values and how preference values 
of diagnoses can be calculated from preference values for single modifications of the 
history. 

In order to compactly represent sets of history-based diagnoses that are all instances 
of one action sequence history-based diagnosis templates can be used which consists 
of that action sequence and a formula constraining the possible instances of the action 
sequence. It is shown that, with an appropriate basic action theory at hand, history- 
based diagnosis templates can be computed taking advantage of the structure of the 
basic action theory’s axioms. We outlined an algorithm guaranteeing to find the most 
preferred diagnosis templates. 

This is work in progress and a lot remains to be done. We already mentioned that we 
intend to refine the computation of diagnosis templates by paying more attention to the 
given observation. The topics currently under investigation also include, e. g., multiple 
observations (at several points in the history), testing and repair (inch sensing). In a 
different framework, these topics are addressed in m in order to characterize diagnostic 
problem solving whereas we also have in mind computational aspects. Another difference 
is that their approach does not allow to consider action variations which we regard as an 
important feature of our approach. 
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Abstract. Making a decision, an agent must consider how his outcome 
can be influenced by possible actions of other agents. A ’best defense 
model’ for games involving uncertainty assumes usually that the oppo- 
nents know everything abont the actual situation and the player’s plans 
for certain. In this paper it’s argued that the assumption results in algo- 
rithms that are too cautious to be good in many game settings. Instead, 
a ’reasonably good defense’ model is proposed: the player should look for 
a best strategy against all the potential actions of the opponents, still 
assuming that any opponent plays his best according to his actual knowl- 
edge. The defense model is formalized for the case of two-player zero-sum 
(adversary) games. Also, algorithms for decision-making against ’reason- 
ably good defense’ are proposed. 

The argument and the ideas are supported by the results of experiments 
with random zero-sum two-player games on binary trees. 



1 Introduction 

Under uncertainty, an agent must consider all the possible situations, called often 
the possible worlds. Although he might not be able to distinguish between many 
of them, the actual ’state of affairs’ severely influences the future course of action 
and the final income the agent is going to gain. 

A two-player poker game may be a good example. The set of possible worlds 
simply consists of all the possible card distributions. Suppose that MAX0has 
*1?AKJ8 ^7 in the actual game. Then MAX can restrict his reasoning to the 
situations that seem plausible to him - namely, all the distributions where MAX 
has ^AKJ8 (see figure Q. In most situations MAX can’t identify MIN’s 
actual beliefs, because he doesn’t know which cards MIN actually possess. Note 
however, that if MAX knew the actual world precisely, he would try to figure 
out the set of situations that seem plausible to MIN, as well as MIN’s future 
line of action. The set would consist of all the distributions where MIN has a 

^ The agent of concern is often called the MAX player (since he should maximize his 
output) in the theory of zero-sum games. His opponent is labeled MIN then - he 
maximizes his output, which means that he tries to minimize the resulting score of 
MAX. 
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Fig. 1. The set of possible worlds 17. In the actual game MAX has '^AKJ8 J|k7, so he 
can restrict the set to Omax- 



particular hand of five cards (in the example on figure 0 497 ^9 <C>10 4J)- Note 
that such a hand must contain no cards possessed by MAX because MAX knows 
that MIN can’t have them. 




MIN 

I A KQ87| 



Fig. 2. The set of plausible worlds according to MIN - if wi is the actual distribution. 

This is the idea underlying the decision-making algorithms gvm and find- 
optimal, proposed in this paper. The player can consider every world from 17 
separately - identifying the possible distributions of resources as well as possible 
beliefs of the opponents. Then he can choose the action that gives him the highest 
expected outcome over all the worlds. 



2 Best Defense 



In Game Theory, a player is assumed to play against optimal defense, since a ’ra- 
tional opponent’ makes always the best decision. For zero-sum games this implies 
that the opponent chooses the worst move from the player’s perspective. A best 
defense model for imperfect information games was proposed in (Frank 1996), 
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(Frank & Basin 1998a) and (Frank & Basin 1998b). It contains the following 
assumptions: 

1. MIN has perfect information about the situation, 

2. MIN knows MAX’s actual strategy, 

3. max’s knowledge is limited to just knowing the set of all possible situations 
(worlds) 17, 

4. the strategy adopted by MAX must be a pure strategy. 

MAX maximizes his expected payoff value (over 17). The opponent (MIN) is 
assumed to be omniscient; thus, he can maximize his payoff directly. 

Since the problem of finding optimal strategy (even in such a simplified set- 
ting) is NP-complete, there is a strong need for suboptimal but less complex 
algorithms. A number of minimaxing algorithms - including vector minimax- 
ing (vm) and payoff-reduction minimaxing (prm) - were proposed in (Frank, 
Basin & Matsubara 1998) and (Frank & Basin 1998b). The ^orithms were 
then compared to the algorithm of Monte Carlo sampling (MC) ,ubased on clas- 
sical minimaxing. In the competition an algorithm was claimed better if it was 
finding strategies close to optimal more frequently than its competitor ~ within 
the notion of ’optimality’ defined above. In a series of experiments on random 
tree games, Monte Carlo sampling algorithm was definitely outperformed by prm 
(and it turned out to be slightly worse than vm, too). However, it’s not clear 
why an algorithm that plays very well against an omniscient opponent should 
also win in a more realistic competition. 

2.1 Experiments with Random Games 

The experiment idea is strongly based on the experiments done by (Frank, Basin 
& Matsubara 1998). The test was conducted for games on complete binary trees 
of depth D. For any of N possible worlds from 17 a payoff is assigned to every 
tree leaf; the payoff may be either 0 or 1. If a game ends in leaf I and world 
w appears to be the actual world, the player wins the payoff value assigned to 
and the opponent loses the same amount. A possible world is described 
with a list of payoffs for all the tree leaves in this world, called a payoff vector. 
Thus, to generate a random game, one must generate a random payoff vector 
for each world. Note that two different worlds can have same payoff vectors. 

An example of such a game is shown on figure 0 

To make this a fair competition between algorithms - the competitors must 
be provided with equal chances of winning. Say the algorithms are called A and 
B. Now, for a particular (randomly generated) game: 

1. first A plays as MAX and B as MIN. The strategies are identified and the 
expected value of payoff computed (over 17); 

2. then the same game is played again - A plays as MIN and B as MAX; 

3. at the end the difference between payoffs is computed. If the difference is 
positive, A wins; if it’s negative then B is the winner. 



^ (Corlett & Todd 1985), (Ginsberg 1999) 
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Fig. 3. A game tree of depth D = 2 with = 4 possible worlds 



1000 games with MAX as a starting player and 1000 games with MIN as 
a starting player were played during every such competition. The algorithms 
involved in the competition were: MC, vm, prm, and the simplest algorithm for 
finding the optimal strategy against ’best defense’ - checking all the possible 
player’s strategies one by one (let’s name it opt-bd, for instance). The output of 
every competition is described by A’s ’triumph supremacy’ (number of rounds 
won by A minus rounds lost by A) and A’s payoff supremacy (the average ex- 
pected payoff value per 1000 rounds). Most experiments were conducted for 
games with D = 8, N = 1000, except the competitions involving opt-bd algo- 
rithm - D = 4, N = 1000 (analyzing any game of more than 4 turns is practi- 
cally infeasible for opt-bd). The results of the actual experiments are shown on 
figure El 





triumph supr. 


payoff supr. 


MC vs. vm 


3.4% 


0.2 


MC vs. prm 


39.1% 


9.2 


vm vs. prm 


33.9% 


8.0 





triumph supr. 


payoff supr. 


MC vs. opt-bd 


36.3% 


7.1 


vm vs. opt-bd 


38.4% 


8.1 


prm vs. opt-bd 


19.1% 


4.2 



Fig. 4. The competition output: triumph supremacy (in [%] of total rounds played) 
and payoff supremacy (per 1000 rounds). Experiment setting: N = 1000 worlds; tree 
depth D = 8 (array on the left), 0 — 4 (array on the right). 



However, the setting of the experiment above may be somewhat misleading 
since we implicitly assumed that both agents have the same knowledge. In real 
situations most agents can at least eliminate some of the worlds from 17 as being 
impossible. For instance, in card games every agent holds some cards in his 
hand. He knows the cards he has, so he can exclude all the card distributions 
inconsistent with this knowledge. Since different players have access to different 
pieces of the reality - the worlds actually possible for MAX {f^MAX C 17) and 
for MIN {f^MiN C 17) should differ in most cases. 

New experiment: MAX and MIN find their strategies with respect to separate 
sets of worlds considered to be possible. f^MAX and f^MiN are generated on 
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random for every game. The players’ knowledge is assumed to be adequate, i.e. 
the output is computed as the expected value of payoff over the worlds from 
^MAX n f^MiN only. Since there is one more random factor in the setting, 20000 
games per round are played instead of 2000. The results are shown on figure 0 





triumph supr. 


payoff supr. 


MC vs. vm 


2.1% 


2.1 


MC vs. prm 


21.7% 


17.7 


vm vs. prm 


21.7% 


14.1 





triumph supr. 


payoff supr. 


MC vs. opt-bd 


18.8% 


12.1 


vm vs. opt-bd 


20.4% 


13.0 


prm vs. opt-bd 


10.6% 


7.0 



Fig. 5. The competition again: players’ belief sets are generated randomly. Experiment 
setting: N — 1000 worlds; tree depth D = 8 (array on the left), D = 4 (array on the 
right). 

The results of the experiments show that it doesn’t have to be beneficial for 
a player to assume that the opponent reaches the upper bound of his theoretical 
capabilities (especially in the context of his knowledge about the actual situ- 
ation). The cautious algorithms: prm and opt-hd were in fact outperformed by 
Monte Carlo sampling, which was considered very suboptimal. A possible reason 
lies in incoherence of the adopted best defense assumptions with the situations 
being encountered in the actual games. 

In perfect information games the opponent (given sufficient resources) plays 
sub-optimally only by his own fault. He can always use the best defense strategy 
since he can find it by minimaxing. The agents can always play best defense 
in perfect information games (just by finding the strategies with minimax). On 
the other hand, in games with incomplete information the opponent is seldom 
able to fulfill the ’best defense’ assumption because his knowledge is insufficient. 
Thus, the model makes the player assume a defense which is impossible to be 
met in most cases. 

3 Reasonably Good Defense 

As the experiments showed, it is not beneficial for the player to overestimate 
capacities of the opponent too much. The best defense model by Frank & Basin 
refers clearly to the worst possible line of events, but this line is quite unlikely 
to occur. 

In a probabilistic framework a model of MIN ’s beliefs is necessary. The model 
should include MIN’s beliefs about the actual situation as well as beliefs about 
the player’s beliefs. The beliefs may depend on the actual world and the state of 
the game. MIN maximizes his expected payoff over 17 with respect to his actual 
state of belief (i.e. he minimizes the payoff for MAX in zero-sum games). MAX 
should maximize his expected payoff over 17 and the set of possible MIN belief 
states. 

Reasonably good defense model: 

If nothing suggests the contrary, the opponent should be as- 
sumed capabilities similar to the player. Thus, MAX’s knowledge 
and skills, and his model of MIN should be symmetrical. 
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In particular - if (in a given situation) no specific knowledge is available 
about likelihood of some possible worlds or different opponent beliefs, equal 
probabilities should be supposed a priori, also when modeling the beliefs of 
other agents. 

3.1 A Simple Case 

The problem is often analyzed in a simplified version, when all the worlds are 
equally probable by rule, but the agents can identify some of them as implausible 
at some point of a game. MAX’s knowledge can be described as Qmax C fi. 
The player can’t know which worlds w G are considered to be plausible by the 
opponent. However, he can restrict the set of possible MIN’s belief sets f^MiN to 
those beliefs which are consistent with his own knowledge - like in the example 
presented on figures Hand Qback in section Q 

To evaluate a MIN node s, MAX may consider evaluations for all the possi- 
ble opponent’s belief sets Umin- (Gamback et al. 1993) present an interesting 
example of such an analysis, concerning Bridge bidding. The player sees his own 
cards (hand) - so he can generate possible distributions from f^MAX by assigning 
the remaining cards to the other players (the authors call each of the alternative 
assignments an R-deat). Every world (R-deal) w S f^MAX determines a MIN’s 
belief set f^MiNiw) ~ namely, Qmin{w) is a set of worlds which cannot be dis- 
tinguished from w by the opponent (in the actual state of the game). In the 
case of Bridge bidding, for instance Qmin{w) consists of all the distributions 
w' in which MIN has exactly the same cards as in w. Whenever MAX needs to 
consider opponent’s decisions he can model the opponent’s view by generating 
^min{w) for each w G ^^max (since analogous function Qmax{w) is needed to 
model the opponent’s knowledge about the player’s possible beliefs, let’s rather 
call the max’s actual belief f^MAX avoid confusion). 

Note that the actual shape of f^MiNiw) depends on the game rules. For 
instance, two players can’t possess the same card in a poker game. So if MIN 
has <C>AI087 in a world w then fiMiN{w) includes all the situations of MIN 
having exactly <C>A1087 ^J, and MAX having none of the cards (the rest of 
the deck must contain none of these cards, too). However, this would not work 
for Canasta, where two complete decks of cards are mixed and dealt - so two 
different players can even possess hands of the same shape! 

An algorithm for finding the decision against ’reasonably good defense’ (2 
players, zero-sum game, no information about worlds’ likelihood except that 
some worlds are actually impossible; the player’s belief doesn’t change when 
moving to another game state - no information flow) is shown on figure 0 Gen- 
eralized vector minimaxing {gvm) is a more universal version of algorithms like 
Monte Carlo or vector minimaxing from (Frank et al. 1988). The algorithm has 
been inspired by ideas from (Carmel & Markovitch 1996) and (Gamback et al. 
1993) - the player looks forward for his opponent’s decision in every possible 
situation, and then maximizes his expected output against such defense. Gvm 
allows to model the players’ knowledge on any arbitrary level, since the func- 
tions Gmini ^max are assumed to mutually encode a player’s beliefs about his 
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gvm (Game, s, player, Worlds); 

Generalized vector minimaxing. Returns the evaluation vector (eval[wi], eval[wn]) 
for node s, together with the player’s chosen move. Parameters: 

Game: game definition, including functions Succ{s) — returning the set of successors for 
node s, Strat{player) — returning the set of all the possible player’s (complete) strategies, 
payoff(l) — returning the MAX’s payoff vector {payoff {l)[wi], payoff {l)[wn]) in leaf I, 
^MlNiw), f2MAx{w) — returning MIN’s and MAX’s beliefs in a particular world w, 
s: game state (node); 

player: the agent who makes a decision at this node (MAX or MIN); 

Worlds: the agent’s actual belief {^max ^MIN this case); 



if Succ{s) — 0 then return {nil, payoff (s))-, 
else: 

■ for every s' € Succ{s) compute Cg/ = (Cs' [wi], ..., e^' [wn]) as: 

r gvm{Game, s' , MIN, nMiN{w))[w] if player = MAX. 

^ \ gvm{Game, s' , MAX, Omax{w))[w] if p/ai/er = MIN 

for every world w £ 

■ if player=M\N then return {s' , Cgi) such that ~Y{,weWorids [^1 minimal, 
else return {s' , Bgi) such that 'Yl-w^worids £s'[H i® maximal. 



Fig. 6. Generalized vector minimaxing. 



opponent’s beliefs as well as his beliefs about his opponent’s beliefs about his 
beliefs etc. 

Note that if 

■ Qmax{w) = ^max opponent knows the player’s state of belief) 

then gvm{Gaine, s, MAX, returns the same strategy as the vector min- 

imaxing algorithm proposed by Frank, Basin & Matsubara. If we also assume 
that 



■ ^min{w) = {tc} (the opponent always knows the actual situation), 

we obtain the instance of vector minimaxing that was actually used in (Frank 
et al 1998). 

On the other hand, if the game definition includes the following assumptions: 

■ Omin{w) = {u>}, 

■ ^MAx{w) = {w}-, 

then gvm becomes equivalent to classical Monte Carlo minimaxing. 

The main disadvantage of gvm is that it’s not always able to find the optimal 
strategy - due to non-locality, a phenomenon observed originally in (Frank 1996), 
and formalized in (Frank & Basin 1998a). The game tree on figure Eldemonstrates 
the phenomenon well. Consider MAX’s decision at node b. If the analysis was to 
be ’local’, MAX would have to prefer the left-hand branch, since his expected 
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payoff equals 0.75 then (against 0.5 when following the right-hand branch). But 
making decision at node b means that MIN has made his decision to move to b, 
not to c, at node a. If we assume that MIN is rational, then his decision must 
make sense - and it makes sense only when he believes that worlds wi,W 2 , W 3 
are irrelevant. In other words, Wi is obviously the only world considered possible 
by MIN at this moment. If we assume that his beliefs are adequate (they can be 
incomplete, but never false), then MAX has to restrict his computation to 1 V 4 
alone, and therefore to pick up the right branch. 

The implication of non-locality is that any ’compositional’ algorithm that 
looks only forward, not backward, is bound to be suboptimal. Thus, the player 
should evaluate his decisions against whole strategies of the opponent, not their 
parts only. On the other hand, it’s not possible to simulate the opponent’s min- 
imaxing over the whole game tree, because this would lead to an infinite loop. 

Figure 7 presents an algorithm for finding the optimal strategy against rea- 
sonably good defense. The algorithm is based on the equilibrium definition for 
zero-sum ga^s. It computes the minmax and the maxmin over the sets of play- 
ers’ strategies, and if they lead to the same result then the optimum has been 
found. Unfortunately, there is often no such an optimum. In this case findoptimal 
returns the minmax strategy, which describes the lower bound of the outcome 
the player can expect (since the assumption that the opponent always knows the 
player’s strategy beforehand defines the upper bound of the opponent’s knowl- 
edge). Another drawback of the algorithm is its computational complexity. 

Note also that if the game definition includes the following assumptions: 

1. flMiNiw) = {w}, 

2 . f^MAx{w) = 

then f indoptimal {Game, MAX, computes the optimal strategy with 

respect to the ’best defense model’ by Frank and Basin. In this sense their 
’best defense’ is a special case of ’reasonably good defense’. Yet the opponent’s 
omniscience is not assumed (in practice) to be an inherent property of games, 
but has to be stated explicitly via flMiN, Gmax functions definition. Moreover, 
the algorithm indicates whether it’s necessary to make any stronger claims about 
the opponent’s knowledge to obtain a solution. 

Finally, it is worth noting that - while gvm, as a minimaxing algorithm, has 
to be suboptimal for games with incomplete information - it can probably be 
improved in terms of accuracy. It demands for some reduction of the impact of 
non-locality on the decision-making process. Frank, Basin and Matsubara has 
already done it for traditional minimaxing within their ’best defense model’ - 
prm algorithm is one of the results. 



3.2 Computational Complexity 

Not surprisingly, opt-bd is highly inefficient since it checks every possible player’s 
strategy - its complexity is doubly exponential on the tree depth and linear on 
the number of worlds (namely, o{N * b^ ) - where b stands for the branching 
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findoptimal (Game, player, Worlds); 

The function returns the player’s chosen strategy. It indicates also whether the strategy 
is optimal, or if it refers to the lower bound of the player’s actual output (when the optimum 
doesn’t exist). The algorithm presented below defines the function for player — MAX. If 
player — MIN., the algorithm is quite analogous. 

[Function output computes the payoff value given a complete strategy pair and a par- 
ticular world from i7.] 



For every str G Strat{M AX): 
m for every w (z Q: let strvect[w] = str; 

m for every w € Worlds: min\w\ = maximize{M I N, strvect, J7m/]v(w)); 

■ let eval[str] = mm[w], w); 

Let stri be that str for which eval[str] is maximal; 

For every strvect G {fi — >■ Strat{M I N)} such that (v G Omin{w) ^ 
strvect{v) = strvect{w)): 

m for every v G Worlds: max[v] = maximize{M AX, strvect, Omax{v)); 

■ let eval[strvect] = Y.,n&woHdsT.,,<zn^jr^(,„)Output{raax[v\, strvect[v],v); 

Let strvect 2 be that strvect for which eval[strvect] is minimal; 

if eval[stn] = maXstr{J2,neworids 

m then return (stri, optimum); 

■ else return (stri, bound); 



maximize (Game, player, strvect. Worlds); 

The function searches the set of all player's strategies, trying to maximize the expected 
payoff value over the given set of possible worlds against given opponent’s strategy vector. 
Of course, MIN player wants to maximize his own payoff, i.e. to minimize the score defined 
by the payoff function. 

Argument strvect is a function of type Q —¥ Strat{M I N) for player — MAX, and 
f2 Strat{M AX) for player — MIN 



if player = MAX then: 

■ return the strategy str G Strat(M AX) for which 

is maximal; 

■ else: return the strategy str G Strat(MIN) for which 

is minimal; 



Fig. 7. Algorithm for hnding the optimal play against reasonably good defense. 
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factor of the game tree) El That’s why suboptimal algorithms are useful: MC and 
vm as well as prm have the time complexity of o{N * b^) (exponential on the 
tree depth, linear on the number of worlds). 

When the opponent’s beliefs are taken into account, the complexity increases. 
The complexity of findoptimal algorithm is o{N^{b ~ )^“''^), so in practical ap- 
plications a suboptimal (but faster) algorithm is necessary. The complexity of 
gvm is exponential on the tree depth and polynomial on the number of worlds: 
o{N^ *b^). The practical complexity of the algorithm may be slightly reduced if 
we assume that players’ beliefs must be adequate: w € Qmin{w), w € Qmax{w) 
for every w. Then the construction of evaluation vectors can be restricted to 
w G Worlds only (instead of all w € Q). 

There is a number of methods that can be used to reduce the search time 
at the expense of its accuracy. Sampling is often used to make the search feasi- 
ble when the number of possible situations is huge; Gamback, Rayner and Pell 
(Gamback et al. 1993) propose such an approach for the case of bridge bidding, 
and Ginsberg’s successful GIB program (Ginsberg 1999) uses Monte Garlo sam- 
pling in the complex domain of bridge card play. Evaluation approximator may 
help to keep the search depth at a reasonable level - Gamback et al. used a 
trained neural network to implement such an approximation function, and they 
reported good results. Also, pruning techniques and heuristic search can be used 
for most domains of application. 



3.3 More Experiments... 

To test the new ideas against existing minimaxing algorithms, random binary 
tree games can be used again. To provide a natural interpretation to the belief 
functions fimiN, ^max every game is treated as an ’imaginary card game’. 
Every world from fl is defined by two hands of c ’cards’ - one hand for each 
player. The deck consists of n ’cards’. A player can play only either the lowest 
or the highest card he possesses at the moment (when it’s his turn to move, 
of course), so at any node (except the leaves) there are exactly two alternative 
decisions that can be made, regardless of the actual hand the player possesses. 
The payoffs for every leaf and each possible world are generated at random. 
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gvm vs. prm 
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gvm vs. opt-bd 
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MC vs. prm 
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vm vs. prm 
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MC vs. opt-bd 


77.2% 
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vm vs. opt-bd 


85.9% 


46.6 
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80.8% 
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Fig. 8. Example results for n = 7, c = 3 cards (tree depth D — 4, N = 140 possible 
worlds) . 

in the case of the experiments here: b — 2. 



3 
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Now, f^MiN{w) is the set of worlds that cannot be excluded by MIN player 
in world w. Namely, it consists of these worlds that assume MIN having the 
hand he actually has. MIN’s beliefs about MAX’s beliefs, Qmax{w) are defined 
in the same way. It is assumed that the players lead their cards secretly, i.e. the 
opponent doesn’t know what card was played exactly - he knows only whether it 
was the highest or the lowest one (the actual world is recognized by both players 
no sooner than at the end of the game). Game parameters are: the tree depth 
D = 2(c — 1), and the number of possible worlds = (") • 

The gvm algorithm is played against other algorithms in a way similar to the 
experiments before. For a particular game, the game is played for every world 
(card distribution), and then the average value of payoff is computed. 1000 games 
are played for every competition: 500 with MAX as the leading player, and 500 
with MIN starting the game (only for n = 8,c = 3, 200 games has been played 
due to complexity reasons). The results (triumph supremacy in [%] of total 
rounds played; payoff supremacy - average/estimated payoff per 1000 rounds) 
are shown on figures 0 and 0 Figure 0 shows also some example results of a 
competition between the traditional algorithms to make the comparison more 
thorough. 





Fig. 9. Triumph supremacy in [%] of total rounds played (left), and Payoff supremacy 
per 1000 rounds (right). 



In every competition gvm appeared to be at least 37.5% better than any 
of the other algorithms (in terms of the triumph supremacy). Moreover, as the 
game complexity increases, gvm starts to win practically 100% rounds. 

The results reveal that algorithms like prm or opt-bd loose less than MC 
or vm when played against gvm. However, when played against each other, the 
previous pattern still holds: MC wins with vm, prm and opt-bd, vm wins with prm 
and opt-bd etc. The reason lies probably in the fact that prm and opt-bd were 
designed to play against a considerably more potent opponent. Thus, playing 
against gvm they can benefit from their cautiousness. On the other hand, MC 
and vm are apparently better off in games against an enemy of the same or 
similar level of skill and knowledge. 
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Another observation may be more surprising. When the tree depth increases, 
the gap between the results of gvm grows rapidly - in terms of the payoff 
supremacy as well as the frequency of winning. However, for a fixed D, gvm 
seems to earn less average payoff if the number of possible worlds increases, 
while still winning more and more rounds. This means that gvm succeeds to find 
a superior strategy for more initial ’hands of cards’, but its expected payoffs for 
every particular hand decrease. The reason is perhaps that when N grows, more 
worlds are possible for any particular hand. Even a good algorithm can’t play 
for all the worlds at the same time, so it is bound to fail in quite a number of 
them. Thus, an average difference in payoffs decrease, although gvm is still able 
to find a strategy better than the others. Moreover, when the tree depth is small, 
there is a very limited amount of different payoff vectors available. Now, when 
the number of worlds being considered at every node increase, it becomes more 
likely that the actual Worlds set may be similar to some of Qmin{w) and/or 
^MAx{w) sets. Which means that gvm minimaxes over similar payoff vectors as 
its competitors in many games. 

3.4 Generalizations 

The games analyzed so far were constrained by several important simplifications. 
More realistic setting should include the following issues: 

— for a game node (state) s: not every move (arc) can be taken in a particular 
situation w € Example: a player can lead A<|k only when he has A<|k; 

— most moves introduce new information. Example: the opponent led A<|k. 
Now, all the worlds in which he hadn’t A<|k can be regarded as impossible; 

— payoff values for a leaf I are defined only in these worlds in which we can 
access the leaf. In fact, the players know the situation (more or less) after 
the last move in many games. Thus - for a particular leaf ~ payoffs in most 
worlds make no sense. 

To incorporate this perspective, the following assumptions can be made: 

— payoff vector is a partial function - payoff {V) : [2 ^ R (values for some 
l,w can be undefined: payoff {l)[w] = undef). It’s good to assume that: 
a + undef = a ■ undef = max{a, undef) = min{a, undef) = a; 

— legal moves are determined by a function Ace. Acc(s) denotes the set of 
worlds in which node s is accessible. Acc can be implemented as follows: 

1. if s is a leaf then Acc(s) = {w G f2 : payoff (s)[w] undef}, 

2. otherwise Acc{s) = Us'gS«cc(^) Acc(s'). 

— every move can reveal some new information, so the beliefs may change as 
the state changes - Qmax, ^min '■ State x Q ^ V{f2). 

The resulting structure resembles in a way the semantics underpinning 
CORA, a complex modal logic for BDI agents proposed in (Wooldridge 2000) - 
with the game tree defining the branching of time, and CImax, ^min standing 
for the belief accessibility relations - although CORA proceeds with qualitative, 
not probabilistic approach to beliefs. 

Next generalization: 
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gvm (Game, s, player, belief); 

Generalized vector minimaxing. Returns the evaluation vector (eval[wi], eval[wn]) 
for node s, together with the player’s chosen move. Parameters: 

Game: game definition, including functions Succ : State 'P(State) — returning the 
set of successors for each node, payoff : State X M — returning MAX’s payoffs, 

BelMi N ,BelMAX ■ State X f2 —y {{f2 —>■ [0, 1]) —>■ [0, 1]) — beliefs about the likelihood of 
particular opponent’s beliefs in a particular situation; 
s : State — the game state being considered; 

player : {MAX, MIN} — the agent who makes the decision at the state; 
belief : ^ [0, 1] — the agent’s actual belief (a probability function); 



if Succ{s) — 0 then return {nil, payoff (s)); 
else: 



for every s' € Succ{s), w G S7, and for every possible opponent’s belief 
oblf simulate the opponent’s minimaxing: 

r hlf T _ f gvm{Game, s' ,MW, oblf)[w] ii player = MAK 

opPa [o J,w\ {gyrn{Game,s',MA'K,oblf)[w] ii player = MIN 

compute the expected payoff for every s' G Succ{s), w G 12 '■ 

_ / Ylobif BeluiNis, w, oblf) ■ oppa' [oblf, w] if player = MAX 

'll. oblf BelMAx{s,w, oblf) ■ oppai[oblf, w] if player = MIN 

if player=MAX then return (s^Cs/) such that belief (w) ■ Cs'fw] is 

maximal. 

else return {s', eg/) such that belief {w) ■ es'[ui] is minimal. 



'M = 



Fig. 10. Generalized vector minimaxing revisited. 



— players may be able to determine some probabilities for the possible worlds 
- not only to tell which worlds are plausible and which implausible now. 
Thus, an actual belief may be a probability function [0, 1] — >■ 17 instead of 
being just a subset of 17; 

— in a given state of a game (and a world) , more than 1 belief may be possible 
within the opponent’s model. This would mean that the player doesn’t know 
the opponent’s reasoning scheme precisely and is bound to guess which belief 
states can result from the opponent’s information analysis. He may also 
consider some of the possible opponent’s beliefs more likely than the others. 

A new version of gvm that takes the new possibilities into account is shown 
on figure mu findoptimal algorithm can be generalized in a similar way. 

The set of all the possible beliefs is in general infinite, so it demands for some 
reduction of the problem. A clever sampling of the set may be a good solution. 

3.5 Dealing with More Capable Opponents 

In the actual experiments, functions like I2min were designed to describe what 
a rational opponent must know in a given situation. On the other hand, the 
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opponent may know (or believe he knows) much more. He can even happen to 
know the whole situation - he may have guessed it from his own card, the MAX 
player’s first move, or even kibitzers’ facial expressions. Frank’s best defense 
assumptions refer clearly to what the opponent can know in the worst possible 
case. 

To deal with such ’possibly more capable’ opponent, the player should con- 
sider every possible opponent’s belief from between what he must and what he 
can know. For the experiment setting (no difference in probability of plausi- 
ble worlds) this would mean maximizing the expected value over all the belief 
sets B such that {w} C B{s,w) C f2MiN{s,w). Thus, to model this situation 
accurately, it is sufficient to assume 

f i if 3 b {ui} C B{s,w) C Qmin{b, w) A p{w) = | l®l ^ ^ 

BeiMlN — \ I else 

I 0 otherwise 

within the input for the generalized version of gvm or findoptimal. ^ 

In the general case (analyzed in the previous section) we can have beliefs 
about opponent’s beliefs defined explicitly with probability functionQSeZM/Af, 
Bel max- Simply, when we suspect the opponent of being more capable than 
just looking at his card and/or the board, it’s good to design the functions so 
that if any opponent’s belief is assumed possible then all the more precise beliefs 
are also assumed possible. 

4 Conclusions 

This paper advocates a thesis that assuming a complete omniscience of the 
opponent may be not quite reasonable in games with incomplete information. 
Instead, the player should optimize his strategy against the expected perfor- 
mance of the other agent (in the mathematical sense) . If the player can identify 
the opponent’s belief for various possible situations, he can do some reasoning 
in the way Gamback, Rayner and Pell showed for the specific case of Bridge 
bidding. Algorithms: gvm and findoptimal implement the idea, and the results 
of the experiments suggest that the ’reasonably good defense model’, proposed 
in this paper, may make sense after all. Of course, the algorithms - especially 
findoptimal - are too inefficient to be used in practice, but they can provide a 
good benchmark for evaluation of suboptimal, faster ones. 

The defense model proposed here - in contrast to the model by Frank and 
Basin - emphasizes the importance of a good information-processing subsystem, 
necessary to acquire and maintain an adequate knowledge about the opponent. 
The actual opponent model may be derived from the game rules or learned by 
the playing agent during the play. The point is that if the player has any (even 
uncertain) information about the agent he plays against available, he should use 
it instead of ignoring it. And if the player has really no information about the 
other agent, he may be better off assuming average capabilities of the opponent, 
rather than capabilities the opponent is unlikely to possess. 



4 



k must represent the number of possible B sets in the equation to keep the probability 
function normalized. 
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Abstract. Many real-life problems can be represented as constraint sat- 
isfaction problems (CSPs) and then be solved using constraint solvers, 
in which labelling heuristics are used to fine-tune the performance of the 
underlying search algorithm. However, few guidelines have been proposed 
for the application domains of these heuristics. If a mapping between ap- 
plication domains and heuristics is known to the solver, then modellers 
can — if they wish so — be relieved from figuring out which heuristic 
to indicate or implement. Instead of inferring the application domains of 
(known) heuristics, we advocate inferring (known or new) heuristics for 
application domains. Our approach is to first formalise a CSP application 
domain as a family of models, so as to exhibit the generic constraint store 
for all models in that family. Second, family-specific labelling heuristics 
are inferred by analysing the interaction of a given search algorithm with 
this generic constraint store. We illustrate our approach on a domain of 
subset problems. 



1 Introduction 

Many real-life problems are constraint satisfaction problems (CSPs), where ap- 
propriate values for the variables of the problem have to be found within their 
domains, subject to some constraints. Examples are production planning subject 
to demand and resource availability, air traffic control subject to safety proto- 
cols, etc. Many of these problems can be programmed as constraint models and 
then be solved using constraint solvers, such as CLp(fd) and OPL H2! 

Constraint solvers are equipped with a search algorithm, such as forward- 
checking, and labelling heuristics, one of which is the default. To enhance the 
performance of constraint models, a lot of research has been made in recent years 
to develop new labelling heuristics, which concern the choice of the next variable 
to branch on during the search and the choice of the value to be assigned to that 
variable. These heuristics significantly reduce the search space HSl- 

However, little is said about the application domains of these heuristics, so 
modellers find it difficult to decide when to apply a particular heuristic, and 
when not. Indeed, there is no universally best heuristic for all instances of all 
constraint models (see, e.g., ^S|), unless NP=P. Thus, we are only told that 
a particular heuristic was “best” for the particular instances used to carry out 
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some experiments with some particular models. Therefore, the performance of 
heuristics is not only model-dependent but also instance-dependent, i.e., for a 
given constraint model, a heuristic can perform well for some (distributions on 
the) instances, but very poorly on others; this is taken into account by some 
generators of model-specific solvers 0na. 

Instead of inferring the application domains of (known) heuristics, we ad- 
vocate inferring (known or new) heuristics for application domains. Obviously, 
the “smaller” an application domain, the “better” its inferrable heuristics. Our 
two-step approach is to first formalise an application domain as a family of 
CSP models, so as to exhibit the generic constraint store for all models in that 
family. Second, the interaction — for a given search algorithm — between the 
constraints in this generic store and the domain propagation during search is 
examined, so as to infer suitable heuristics for any model in that family. Due to 
the instance sensitivity of heuristics, the outcome of this process usually is a set 
of heuristics, rather than a single one. In this paper, we illustrate this approach 
on a domain of subset problems. 

If a mapping between application domains and heuristics is known to the 
solver, then modellers can — if they wish so — be relieved from the procedural 
aspect of modelling, namely figuring out which heuristic to indicate or imple- 
ment. Forcing modellers to deal with this procedural aspect may not only add 
a challenging step but also has the disadvantage that they must commit — at 
modelling time — to a single heuristic and thus expose their models to the in- 
stance sensitivity of heuristics. In companion work PH], we address the issue 
of selecting or switching — at solving time — among the inferred family-specific 
heuristics resulting from our approach, according to the instance to be solved. 
Our ultimate aim is thus a new generation of more intelligent solvers that allow 
CSP modellers to concentrate on the declarative aspect of modelling, without 
compromising (much) on efficiency. 

This paper is organised as follows. In Section E) we introduce the notion of 
family of CSP models as a formalisation of an application domain. We illus- 
trate this with a domain of subset problems and exhibit a generic finite-domain 
constraint store of a family for this domain. Then, in Section 0, we present our 
analysis of this generic constraint store, infer two labelling heuristics, and show 
our initial empirical results. Finally, in Section 0 we conclude, compare with 
related work, and discuss directions for future research. 



2 CSP Model Families 

Informally, an application domain is a set of “related” CSPs. For instance, in the 
SUBSET domain, a given number of elements have to be selected from a given 
finite set such that any two of them satisfy some constraint p. In this domain, 
CSPs are related in the sense that the actual constraint p differs between them. 
Sample CSPs in this domain are finding a clique of a given size within a given 
graph (where p requires that any two vertices of the clique be connected by an 
edge of the graph) and finding an independent subset of a given size among 
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the vertices of a given graph (where p says that any two vertices of the subet 
must not be connected by an edge of the graph) . Application domains of coarser 
granularity are scheduling, configuration, resource allocation, and so on. 

For a given constraint modelling language, a CSP model family is an open 
CSP model in that language, ‘open’ in the sense that some of its (predicate or 
type) symbols are neither primitive to the language nor defined in the model. An 
actual CSP model is closed, in the sense that all its symbols must be primitive 
or defined. From a model family, a model can thus be obtained by substituting 
closed types and closed formulas for all its open symbols, and possibly by adding 
parameters. Model families can be used to formalise application domains. There 
are in general several ways of formalising a domain as a model family, in a given 
language, namely depending on the chosen data modelling. An instance of a 
CSP model M is obtained from M by replacing all its formal input parameters 
by actual values and dropping the universal quantifications on these parameters. 
An instance of a model is thus also a model, albeit without input parameters. 

Example 2.1. Assume CSP models are written in a very expressive, purely 
declarative, typed, set-oriented, first-order logic constraint modelling language, 
such as our ESRA ftiH) . which is designed to be higher-level than even OPL \n\ 
(We can automatically compile jSj ESRA programs into lower-level languages 
such as OPL.) Since ESRA has set variables (unlike OPl), the following (sugared 
version of an) ESRA model family is a candidate formalisation of the SUBSET 
domain: 

VT, S : set{a) . Vfc : int . {subset{T~^ , fc+, S) S C T A size{S, k) A 

: a . {ti G S A tj G S A ti tj ^ pfti, tj))) { u se ) 

where the superscript designates the input parameters. In words, sets S and 
T of elements of type a are in the subset/ 3 relation with integer fc iff S' is a set 
of k elements from T, such that any two distinct elements ti and tj of S satisfy 
constraint p. The only open symbols are type a and constraint p, as size, C, G, 
and yf are primitives of ESRA, with the usual meanings. From the Subset model 
family, we can obtain the following (sugared) ESRA model: 

VV, C : set{int) . Vfc : int . \/E : set{int x int) . 

{cliquek{{V~^ , £’+), fc+, C) GA C Q V A size{C, k) A (cliquek) 

Vvi,Vj : int . (vi G C A Vj G C A Vi ^ Vj -A {vi, Vj) G E)) 

It is a model for finding a clique C of an undirected graph (given through its 
integer vertex set V and edge set E), such that the clique has fc vertices. 



Example 2.2. At a lower level of expressiveness, say when set variables are not 
available (such as in CLp(fd) 0 and OPL ini)> the usual representation of an 
unknown subset S' of a given finite set T (of n elements) is a mapping from 
T into Boolean variables (in {0,1}), that is one conceptually maintains n cou- 
ples {ti,Bi) where the (initially non-ground) Boolean Bi expresses whether the 
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(always ground) element U of T is a member of S or not0 

yti : a . ti G T ^ {Bi ti G S) (1) 

This Boolean representation of set variables consumes more memory than the 
set-interval representation of CON junto 0 and OZ uni, but both have been 
shown to create the same 0(2") search space 0. 

Given this Boolean representation of the sought subset S, restricting its size 
to k can be expressed as the following n-ary constraint: 

n 

Y^B, = k (2) 

1=1 

Let us also look at the remaining part of SUBSET, which requires that any 
two distinct elements of the subset S of T must satisfy a constraint p. Formally 
(using the sugared ESRA syntax again, for the sake of symbolic reasoning): 

S CT A Wti, tj : a . U G S Atj G S Ati ^ tj ^ p{U, tj) 

This implies 

Vti, tj : a.tiGT Atj GT AtiG S Atj G S Ati^tj ^ p{ti, tj) 

which is equivalent to 

Vfi, tj : a . ti GT Atj GT Ati ^ tj A ~^p{ti, tj) -A ~'{ti G S Atj G S) 

By O, this can be rewritten as 

Vti, tj : a . ti G T A tj G T A ti ^ tj A ~'p{ti, tj) -A ~<{Bi A Bj) (3) 

The sugared version of an OPl/clp(fd) model family formalising the SUBSET 
domain thus consists of constraints 0 and O; we denote it by Subsets- For 
any two distinct elements ti and tj of the given set T, with Boolean variables B^ 
and Bj , if p{ti , tj ) does not hold, the following binary constraint arises: 

^{B, A B,) (4) 

It is crucial to note that the actual finite-domain constraints are thus not in 
terms of p, hence p can be any formula. Therefore, the generic finite-domain 
constraint store for any instance of any model of the Subsets family is over a set 
of (only) Boolean variables. It contains an instance-dependent number of binary 
constraints of the form (^, as well as the (always unique) n-ary constraint (EJ. 

As the set-interval representation of set variables does not allow the definition 
of some (to us) desirable high-level primitives, such as universal quantification 
over elements of non-ground sets, the set variables of ESRA (see Fxa,mn]e l2.1 ll are 
compiled m using the Boolean representation of Fxa,mn]e l2.2l In the remainder 
of this paper, our approach to inferring labelling heuristics from an application 
domain is illustrated on the SUBSET domain, and we (thus) focus on its Boolean 
modelling in the Subsets family. 



^ In formulas, we use atom Bi as an abbreviation for Bi — 1. 
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3 Inferring Labelling Heuristics 

It is known that the order in which the variables are considered for instantia- 
tion, and the order in which the values are attempted for assignment to variables 
during search have a substantial impact on the number of backtracks performed 
and the time taken by a search algorithm to solve a CSP model. Deciding on 
these orders is the objective of labelling heuristics. We now infer some labelling 
heuristics for the SUBSET domain by examining the domain propagation per- 
formed on the generic constraint store — for the Subsets family — by a search 
algorithm during labelling. For the sake of illustration, we here choose the for- 
ward checking (FC) algorithm, which is used in many solvers. It works as follows: 
Whenever a variable is labelled by a value v, the values of the future variables 
that are inconsistent with v are removed from the domains of these variables. 

In Section lO we present our analysis of the obtained generic constraint 
store. Next, in Section E21 we infer some FC labelling heuristics for Subsets 
models. Finally, in Section E3 we report on our initial experimental results. 

3.1 Analysis of the Generic Constraint Store 

We analyse the generic constraint store using the values n (the size of the given 
set T, hence the number of Boolean variables involved) and k (the given size of 
the sought subset S). In models of the Subsets family, each Boolean variable Bi 
in {Bi, . . . , Bn} is at any moment associated with the set Vi of still unassigned 
variables Bj (where 1 < J < n) that constrain Bi with a binary constraint of 
the form (0. A binary constraint of this form requires that the variables Bi and 
Bj cannot simultaneously be assigned 1. Furthermore, the n-ary constraint (0 
restricts all the variables such that k of them must be assigned 1. Let ko (resp. 
ki) be the current number of variables that have yet to be assigned 0 (resp. 1). 
Initially (before the labelling), fco = n—k and k\ = k. During labelling, the values 
of ko and ki decrease because of the assignments and propagation. If either ko or 
fci reaches 0, the propagation caused by the n-ary constraint forces the other one 
to also become 0. Therefore, at the end (after the labelling), fcg = fci = 0. Note 
that the mathematical variables Vi . . .Vn, ko, k\ are only explanatory devices, 
but not actually stored and manipulated anywhere. 

We now monitor the FC propagations triggered by the assignment of values 
(from {0, 1}) to the Boolean variables. The ordering of the variables and values 
is irrelevant in this analysis: suitable labelling heuristics will be inferred in Sec- 
tion [1.21 When fco > 0 and fci > 0, we consider two cases, namely Case A, the 
assignment of 0, and Case B, the assignment of 1 to the chosen variable, say Bi. 

Case A. li Bi is assigned 0, the current number of variables that have yet to be 
assigned 0 is decremented by 1, so fcg becomes fcg — 1. Two sub-cases arise now: 

— If fcg = 0 now, then all the fci yet unassigned variables are assigned 1 during 
propagation due to (only) the n-ary constraint 0, leading to fci = 0 also. 
Now exactly n — k variables have been assigned 0 and k variables have been 
assigned 1. However, if there is a binary constraint of the form 0 between 
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any two of these fci variables, then this assignment fails, which leads to 
backtracking. Otherwise, this assignment succeeds. 

— If fco > 0 still, then, for all v S Vi, the domain of v remains the same, because 
the assignment of 0 to any variable in a binary constraint of the form 
always succeeds without propagation. 

By the instantiation of a variable by 0, there is thus a possibility of backtracking 
only if fco reaches 0, because the assignment may fail. 

Case B. If Si is assigned I, the current number of variables that have yet to be 
assigned 1 is decremented by 1, so fci becomes fci — 1. Two sub-cases arise now: 

— If fci = 0 now, then all the fcg yet unassigned variables are assigned 0 during 
propagation due to (only) the n-ary constraint Q, leading to fco = 0 also. 
Now exactly fc variables have been assigned 1 and n — k variables have been 
assigned 0, without violating any constraints. Indeed, as seen in Case A, the 
assignment of 0 to a variable fails only if fco becomes 0 and there is a binary 
constraint between any two of the fci variables. However, there are here 
no unassigned variables left, as fci = 0 already. Therefore, this assignment 
always succeeds. 

— If fci >0 still, then, for all v G Vi, the variable v is assigned 0 during 
propagation because of the binary constraints of the form (0. Thus, fco 
becomes fco — \ Vi\. The new value of fco now gives rise to the following sub- 
sub-case analysis: 

• If fco < 0 now, then one of these assignments must fail and immediate 
backtracking occurs. 

• If fco = 0 now, then all the fci yet unassigned variables are assigned 1 
during propagation, leading to fci = 0 also. As seen in Case A, if there is 
a binary constraint of the form between any two of these fci variables, 
then this assignment fails, which leads to backtracking. Otherwise, this 
assignment succeeds. 

• If fco > 0 now, then this assignment succeeds. 

By the instantiation of a variable by 1, there is thus a possibility of backtracking 
only if fco reaches 0 first. Should fco become negative, the assignment fails, and 
thus an immediate backtracking occurs. On the other hand, the assignment 
always succeeds if fci reaches 0 first. 

It is very important to notice that Case B may include Case A. On the 
other hand. Case A never includes the general situation of Case B. Therefore, 
the analysis became of finite size and complete, as there is no case where it is 
impossible to exactly foretell all propagations! 

3.2 Inference of Heuristics 

In models of the Subsets family, the assignment — under FC search — of 0 to 
a Boolean variable triggers propagation only when fco reaches 0, and this inde- 
pendently of the order of the variables being instantiated by 0 so far. Therefore, 
if the set of variables that will be assigned 0 is not chosen carefully (e.g., when 
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there are no binary constraints between them, in which case there probably 
are binary constraints between the other variables), backtracking is unavoidable 
once fco reaches 0. The only way to avoid backtracking is to choose the right 
set of n — k variables that are assigned 0. However, finding such a subset of the 
Boolean variables is itself a subset problem. 

The assignment of 1 to a variable is noteworthy because every assignment 
caused by propagation upon fci = 0 succeeds, so that no backtracking can hap- 
pen. Also, the order of the variables being assigned 1 is quite important because 
it can significantly affect the decrease in fep- Indeed, as seen in Case B, if fci > 0 
still, then fco becomes fco — \ Vi\. The variable Bi being assigned 1 is associated 
with a set Vi (the set of the still unassigned variables that constrain Bi) that 
thus directly affects the decrement in ko. If the variables being assigned 1 are 
ordered in a way that they do not cause much decrease in fcg, then backtracking 
when fco < 0 and any possible backtracking when fcp = 0 are delayed. Backtrack- 
free assignment is thus guaranteed by allowing fci to reach 0 first. However, 
backtrack- free assignment is not guaranteed if it is fco that reaches 0 first. 

We can thus infer the following two labelling heuristics from the previous 
considerations: 

— If there is at least one solution, we should instantiate some variables by 
1, and try to keep each \Vi\ as small as possible if we want fci to reach 0 
first (which leads to backtrack- free assignment). Thus, during FC search, if 
we choose a variable that is participating in the smallest number of binary 
constraints, then we force fci to become 0 before (or at the same time) as fco 
does, because, by this way, we achieve a small decrease in fco. This heuristic 
can be seen as an instance of the succeed- first principle. 

— If there is no solution, then it is impossible to reach the state fci = 0. 
Search effort can then be saved by forcing the search to reach a state with 
definite backtracking (when fco < 0) or possible backtracking (when fco = 0) 
as soon as possible. Thus, during FC search, if we choose a variable that is 
participating in the largest number of binary constraints, then we force fco to 
be negative or to become 0 before fci does, because, by this way, we achieve 
a big decrease in fco. The value ordering is thus irrelevant. This heuristic can 
be seen as an instance of the fail-first principle. 

As it is initially unknown whether there is a solution or not, it is very difficult 
to choose which of these two heuristics to use in order to guide the search pro- 
cess. This paper is only concerned with the inference of heuristics; the issue of 
deciding when to use which one, or when to switch between them, is addressed 
in companion work Em]. 

Following these considerations, we implemented the following static labelling 
heuristics, namely in SICSTUS CLp(fd) (which has an FC solver): 

— Hf, which chooses the variable that is constraining the smallest number of 
variables, and assigns the value 1 first. 

— (resp. Hi), which chooses the variable that is constraining the largest 
number of variables, and assigns the value 0 (resp. 1) first. 
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Being static, these labelling heuristics choose a variable that is initially con- 
straining the smallest/largest number of variables. Note that this implementa- 
tion of the heuristics is our choice, but that the heuristics could be implemented 
in another way, say by re-ordering the variables at solving-time. Investigation 
of the superiority or the inferiority of such dynamic variable orderings, which 
choose a variable that is constraining the smallest/largest number of the future 
(yet unassigned) variables, to the static ones is left as future work. 

3.3 Experiments with the Heuristics 

Experimental Setting. We measured the cost (in CPU time and in number of 
backtracks) of our heuristics on a very large number of instances of the models 
of the Subsets family. These experiments confirmed the anticipated strengths 
and weaknesses of the heuristics, which are exploited in our companion work on 
deciding when to use which heuristic, or when to switch between them CHI]- 

For binary CSPs, a clasfl of instances is usually characterised by a tuple 
{n,m,pi,p 2 ), where n is the number of variables, m is the (assumed constant) 
domain size for all variables, pi is the (assumed constant) constraint density, and 
P 2 is the (assumed constant) tightness of the individual constraints. Experiments 
are then conducted by iterating over an interval of instance classes and generating 
a suitably sized sample of random instances for each class. For each sample, the 
median or average solving cost is computed. 

However, our generic constraint store features a non-binary constraint, so we 
cannot literally apply this characterisation of instance classes. In any case, the 
latter has been criticised because it is unrealistic to have a constant tightness 
P 2 for all constraints, so that many possible instances can never be generated. 
For these two reasons, we developed the following characterisation of instance 
classes, which is specific to the considered family. It is not subject to any of the 
criticisms in [Q , because it exploits the structure of the generic constraint store. 

The generic finite-domain constraint store for the Subsets family is parame- 
terised by the number n of Boolean variables involved (i.e., the size of the given 
set T) and the given size k of the sought subset S, and contains an instance- 
dependent number b of binary constraints of the form ®. The number n of 
variables and the density p\ of the constraints are kept from the previous char- 
acterisation, with Pi being ti^re. The domain size m is dropped, as it 

always is 2, because we need only consider the Boolean domain {0, 1}. Since the 
considered binary constraints are of the form -<{Bi /\ Bj), their tightness always 
is 3/4 and thus does not become a parameter. The tightness of the n-ary con- 
straint however is (^) /2", and thus varies with n and k. As we already use n, the 
size k becomes the final parameter in our characterisation of instance classes, 
which is thus summarised by the triple (n, p\,k). 

For the purpose of this paper, we generated random instances in a coarse 
way, by not considering all possible values of n up to a given limit. The number 
n of variables ranged over the interval 10. .120, by increments of 10. We varied 

A class (of instances) is not to be confused with a family (of CSP models). 
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the density pi over the interval by increments of 0.1. The values of k 

ranged over the interval l..n, by increments of 1. Considering the sizes of these 
intervals, the number of our experiments was huge and their execution was very 
time-consuming. Given more time, instances generated in a more fine-grained 
way could be used instead and help to make our (future) results more precise. 
Our objective here only is to show the heuristics in action, but not to provide 
the most detailed statistics for our companion work on deciding when to use 
which heuristic, or when to switch between them M- 

Rather than only comparing the inferred heuristics to each other, we also 
compared them to some others. For time reasons, we restricted ourselves to the 
following two additional heuristics: 

— Hg, which chooses the variable that is constraining the smallest number of 
variables, and assigns the value 0 first. 

— Default, the default labelling heuristic of SICSTUS CLp(fd), which labels 
the leftmost variable in the provided sequence of variables, and the domain 
of the chosen variable is explored in ascending order (i.e., 0 first in our case). 

The heuristic 7?° is a natural complement to the inferred heuristics, and was also 
implemented in SiCSTUS CLp(fd). In the absence of a labelling heuristic provided 
by the modeller, each solver uses its default heuristic. Since our experiments were 
conducted in SICSTUS CLp(fd), its default heuristic had to be used here. (The 
experiments thus have to be repeated for each FC solver, because their default 
heuristics change.) 

If a combination of the inferred heuristics beats — on the average over numer- 
ous instances of the family — the default heuristic of the solver, then this com- 
bination can become a family-specific and even highly instance-sensitive default 
heuristic of the solver. The determination of such a combination is addressed in 
our companion work If this idea is repeated for other families, then the 

modellers can — if they wish so — be relieved from the procedural aspect of 
modelling and even be protected from the instance sensitivity of their heuristics. 

Our experiments were made over random instances (of models) of the con- 
sidered family for the following reason. Towards using real-life instances, we 
would have had to first pick some models within the considered family, but we 
would then have been unable to justify why these models were picked rather 
than some others. The purpose of our experiments uni was to generate statistics 
that guide us in our companion work where we aim at a family-specific 

default heuristic for a solver, which must be able to handle random instances 
over that entire family. We do not aim at a heuristic for a specific model, which 
would have to be able to handle (only) real-life instances of (only) that model. 
Experiments. Having thus chosen the intervals and increments for the param- 
eters in our characterisation of an instance class, we randomly generated many 
different instances and then used the 5 heuristics in order to solve them or prove 
that they have no solutions. Some of the instances were obviously too difficult 
to solve or disprove within a reasonable amount of time. Consequently, to save 
time in our experiments, we used a time-out (of 3,600,000ms) on the CPU time; 
upon time-out, the current number of backtracks was recorded. 
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(a) Pi = 0.1 (b) Pi = 0.5 

Fig. 1. CPU-time (in ms) in terms of k for the 5 heuristics on n = 100 



In order to analyse the effects of each heuristic on different instances, we drew 
various charts, for example by keeping n and pi constant and plotting the median 
costs of the samples for each k. Figure G] shows an example of the behaviours 
of the 5 heuristics in terms of CPU-time on the instances where n = 100, with 
Pi — 0.1 and Pi — 0.5, respectively. FigureEl shows their behaviours in terms of 
the number of backtracks on the same instances. 

These figures do not show that the generated instances exhibit three very 
interesting regions in terms of k, no matter what n and pi are: up to some 
value V of k, all instances have a solution; then, until some other value w of fc, 
some instances have a solution and some do not; beyond w, all instances have 
no solution. A visible interesting observation is that, without a time-out, the 
solving-times for instances increase with k until some point, whereupon they 
decrease. With the heuristics we used, we recorded time-outs in all three of 
the mentioned regions. After taking the median cost of the generated sample of 
random instances for each class {n,pi,k), we observed three different zones in 
terms of k: up to some value j in 0..n, the instance with the median cost has a 
solution; from some other value Hn j -|- 1 .. n+1, the instance with the median 
cost has no solution; in-between, the instance with the median cost timed out. 
It is in general unknown where j and I are compared to v and w. The values of 
j, I, u, w depend on n and pi. 
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The position of k relative j and I yields the following analysis of the be- 
haviours of the heuristics in terms of the CPU-time they take (see Figured): 



Over l..j, the heuristic always finds a solution, in mostly constant CPU- 
time. Default performs the best until k reaches some d in 0..j, where d is 
small. However, over d -|- 1 .. j, the heuristic outperforms Default. The 
heuristics and id/ perform as well as Hf until k reaches some i in l..j. 
However, over i + 1 .. j, the heuristic id/ outperforms id/ and id/. Heuristic 
id/ usually has the worst performance. In conclusion, over l..j, the heuristic 
id/ is the best over d -|- 1 .. j, with l..d being always a very small interval. 
The range of k where id/ performs the best varies in size with respect to pi, 
given n: compare Figures 1(a) and |l(bj| 

Over j + 1 .. Z — 1, we cannot compare the heuristics because they all timed 
out. This can be observed in Figure |l(a)|for k in 34. .37. 



Over l..n, the heuristic id/ always proves that there is no solution, in de- 
creasing CPU-time. Heuristic id/ usually has the worst performance. In this 
range, the heuristic id/ is always outperformed by id/ and id/, and performs 
as badly as id/. The heuristics id/ and id/ perform the best until k reaches 
some i in l..n, whereupon Default outperforms all the others. The range of 
k where id/ and id/ , or Default perform the best varies in size with respect 
to pi, given n: compare Figures [l(a)| and|l(b)| 



The heuristic id/ mostly performs the best when there is an observed solution. 
This can easily be explained by the fact that it was designed to try and find a 
solution, while assuming there is one. The heuristics id/ and id/ mostly perform 
the best when there is no observed solution. This is because they were designed 
to prove that there is no solution, while assuming there is none. The reason why 
Default sometimes outperforms the other 4 heuristics is that it has no solving- 
time overhead. Somewhere in j -|- 1 ^ — 1, a phase transition from the soluble 

region to the non-soluble region occurs, and all the heuristics failed to efficiently 
handle these instances and thus timed out. 

The position of k relative j and I yields an analysis of the behaviours of the 
heuristics in terms of the number of backtracks they make (see Figure EJ: 



— Over l..j, the heuristic 7d/ always finds a solution, mostly in 0 backtracks. 
Default always performs worse than id/. The heuristics id/ and Hi initially 
perform as well as id/, but start backtracking earlier. Heuristic id/ usually 
has the worst performance. In conclusion, over l..j, the heuristic id/ is always 
the best. The range of k where id/ performs 0 backtracks varies in size with 
respect to pi, given n: compare Figures |2(a)| and f2(b)| 

— Over j + 1 .. / — 1, we cannot compare the heuristics because they all timed 
out. This can be observed in Figure |2(^ for k in 34. .37. 

— Over l..n, the heuristic id/ always proves that there is no solution, in decreas- 
ing numbers of backtracks. Heuristic id/ usually has the worst performance. 
In this range, the heuristic id/ is always outperformed by id/ and id/, and 
performs as badly as id/. The heuristics id/ and id/ perform the best until 
k reaches some i in l..n, whereupon all the 5 heuristics perform the same 
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(a) Pi = 0.1 (b) Pi = 0.5 

Fig. 2. Number of backtracks in terms of k for the 5 heuristics on n = 100 



number of backtracks. The range of k where and Hi (resp. all the 5 
heuristics) perform the best (resp. the same) varies in size with respect to 
Pi, given n: compare Figures p(a)| and [2(b)[ 

The heuristic always performs the best in number of backtracks (and mostly 
with 0 backtracks) when there is an observed solution, because it was designed 
to try and find a solution, while assuming there is one. The heuristics and 
Hi mostly perform the best in number of backtracks when there is no observed 
solution. This is because they were designed to prove that there is no solution, 
while assuming there is none. Somewhere in j + 1 — 1, a phase transition from 

the soluble region to the non-soluble region occurs, and all the heuristics failed 
to efficiently handle these instances and thus timed out. 

4 Conclusion 

Labelling heuristics may lead to a substantial reduction of the search space when 
solving CSP models. However, little is known about the application domains of 
the known heuristics. This work follows the call of Tsang et al. for mapping com- 
binations of algorithms and heuristics to application domains m- Rather than 
inferring the applications domains of (known) algorithm/heuristic combinations. 
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we here advocate inferring (known or new) algorithm/heuristic combinations for 
application domains. 

Our approach is to first formalise a CSP application domain as a model 
family, so as to exhibit the generic finite-domain constraint store for all models 
in that family. By analysing the interaction of an algorithm with this generic 
constraint store, one can then infer labelling heuristics for that family. Usually, 
one would at least look for a heuristic that excels at finding the first solution, 
one that excels at disproving the existence of solutions, and one that detects and 
handles the phase transition. We here illustrated this approach on a domain of 
subset problems, as well as on the effect of labelling heuristics for a fixed search 
algorithm, namely forward checking. We inferred two heuristics for this domain, 
one for each of the first two kinds. 

We generate random instances by iterating over an interval of {n,pi,k) in- 
stance classes and generating a suitably sized sample of random instances for 
each class. For each sample, if the instances are comparable (e.g., all the instances 
have a solution), the median cost is computed; otherwise (e.g., some instances 
have a solution but some do not), we cannot judge which heuristic is the “best” 
for this sample. We then devise a lookup table, where either the “best” heuris- 
tic for a given instance class (n,pi, k) is designated j7], or a switching between 
heuristics is designated because none of the heuristics is considered to be better 
than another one for this class of instances dH- This switching can be done by 
deploying one of the heuristics first, and monitoring the progress so as to switch 
to the next one in case of thrashing. This lookup table is then used by a meta- 
heuristic. If this meta-heuristic beats — on the average over numerous instances 
of the family — the default heuristic of the solver, then this meta-heuristic can 
become a family-specific and even highly instance-sensitive default heuristic of 
the solver. If this is repeated for many application domains, then modellers can 
— if they wish so — be relieved from indicating or implementing a heuristic 
at modelling-time, which often is a too early commitment anyway, due to the 
instance-sensitivity of heuristics. 

In terms of related work. Figure 0 shows the classical approach to designing 
heuristics in full lines, whereas the contribution of our approach is emphasised in 
dashed lines and italicised text. A curved arrow from a full line to a dashed line 
indicates our replacement of the full line with the dashed line. We thus replace 
the design of a single heuristic for a CSP model in the presence of a solver (i.e., 
search algorithm) with the inference of a set of heuristics for a model-family 
by analysis of the propagation performed by that solver on the family-specific 
generic constraint store during labelling. Also, in our approach, random instances 
are generated only for the considered family (which does not necessarily contain 
binary CSPs), rather than for arbitrary (binary) CSPs. 

Closely related to our work is first Minton’s multi- TAG system H2|, which 
automatically synthesises an instance-distribution-specific solver, given a high- 
level model of some CSP and a set of training instances. While MULTI-tac uses 
a synthesis-time brute-force approach to generate candidate problem-specific 
heuristics from a set of heuristics described by a grammar, we propose inferring 
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Fig. 3. Contributions to the classical approach to designing heuristics 



candidate family-specific heuristics manually by analytically reasoning about 
the generic constraint store of the family. Second, Sadeh and Fox propose a 
probabalistic framework for the job shop scheduling domain so as to capture the 
search space. Based on this framework, a domain specific heuristic is derived 
m- The derived heuristic significantly reduces the search space of the instances 
used in the experiments. However, the instance sensitivity of heuristics is not 
tackled, and only one heuristic is derived for the domain. 

Our future work includes investigating the superiority or the inferiority of 
dynamic variable orderings, which choose a variable that is constraining the 
smallest /largest number of the future (yet unassigned) variables, to the here 
investigated static variable orderings, which choose a variable that is initially 
constraining the smallest/largest number of variables. 

We are also planning to investigate other application domains, such as m- 
subset problems (where a maximum of m subsets of a given set have to be found, 
subject to some constraints), relation problems (where a relation between two 
given sets has to be found, subject to some constraints) 0 , permutation problems 
(where a sequence representing a permutation of a given set has to be found, 
subject to some constraints) [0|, and sequencing problems (where sequences of 
bounded size over the elements of a given set have to be found, subject to some 
constraints) |S|, or any combinations thereof. 

All results will be built into the compiler of our ESRA constraint modelling 
language which is more expressive than even OPL ini. This will help us 
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to fulfill our design objective of also making ESRA more declarative than OPL, 
without compromising (much) on efficiency. 
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Addressing the Qualification Problem in FLUX 
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Abstract. The Qualification Problem arises for planning agents in real- 
world environments, where unexpected circumstances may at any time 
prevent the successful performance of an action. We present a logic pro- 
gramming method to cope with the Qualification Problem in the action 
programming language Flux, which builds on the Fluent Calculus as a 
solution to the fundamental Frame Problem. Our system allows to plan 
under the default assumption that actions succeed as they normally do, 
and to reason about these assumptions in order to recover from unex- 
pected action failures. 



1 Introduction 



Intelligent agents in open environments inevitably face the Qualification Prob- 
lem: The executability of an action can never be predicted with absolute cer- 
tainty; at any time, actions in the real-world may surprisingly fail |1IH. Yet it 
would be irrational, and even impossible in general, for a planning agent to fore- 
see all conceivable reasons for an action to go wrong. Rather, a rational agent 
needs to devise plans under the assumption that the world will behave as ex- 
pected. On the other hand, being aware of these assumptions helps an agent to 
explain and recover from unexpected failures encountered during the execution 
of a plan. 

For a long time, the main theoretical result on the Qualification Problem 
had been a negative one: While a solution must involve the ability to assume 
away, by default, so-called abnormal qualifications of actions m, straightfor- 
ward minimization of abnormality yields anomalous models Pj. This problem 
being unsolved, previously developed action programming languages and plan- 
ning systems, such as |8I1 611 21^ . did not attempt to address the Qualification 
Problem. The problem of anomalous models has, however, recently been solved 
in a formal account of the Qualification Problem presented in |2I]. This the- 
ory builds on the Fluent Calculus as a predicate logic formalism for reasoning 
about actions which is one of the standard solutions to the fundamental Frame 
Problem m- 

In this paper, we integrate the theoretical account of the Qualification Prob- 
lem into the action programming language Flux (the Flu ent Calculus Executor) 
Based on constraint logic programming. Flux allows to specify and reason 
about actions with incomplete states, and thus to solve planning problems under 
incomplete information. Its core consists of a logic programming account of the 
Fluent Calculus solutions to the Frame and Ramification Problem 1 1 t)l 1 7j . Ex- 
tending Flux so as to cope with the Qualification Problem, our system allows 
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the user to specify default assumptions concerning the executability and effects 
of actions. Plans are then generated under these assumptions, and the system is 
able to reason about the assumptions made and to withdraw appropriate ones in 
order to explain and recover from unexpected action failures. The language al- 
lows to distinguish between strong qualifications (actions not being executable) 
and weak ones (actions producing unexpected effects). Furthermore, it supports 
the specification of preferences among the default assumptions, by which is aided 
the search for reasonable explanations in case of unexpected action failure. 

The paper is organized as follows. In Sectional we recapitulate the theoretical 
account of the Qualification Problem in the Fluent Calculus. In Section 0 we 
show how to extend Flux to cope with the Qualification Problem. For a more 
detailed discussion of this section the reader is referred to In Section E] an 
application is described and Section 0 gives a summary. 

2 The Qualification Problem in the Fluent Calculus 

2.1 Simple State Update Axioms 

The simple Fluent Calculus m combines, in pure classical logic, the Situation 
Calculus with a STRIPS-like solution to the representational and inferential 
Frame Problem. The standard sorts ACTION and sit (i.e., situations) are inher- 
ited from the Situation Calculus [Z| along with the standard functions S'o : SIT 
and Do : ACTION x sit i— 7 > sit denoting, resp., the initial situation and the 
successor situation after performing an action; furthermore, the standard pred- 
icate Pass : ACTION X SIT denotes whether an action is possible in a situation. 
To this the Fluent Calculus adds the sort state with sub-sort fluent along 
with the pre-defined functions 0 : state, o : state x state i— >■ state, and 
State : SIT >->■ state, denoting, resp., the empty state, the union of two states, 
and the state of the world in a situation. Based on this signature, the Fluent 
Calculus provides a rigorously logical account of the concept of a state being 
characterized by the set of fluents that are true in the state. To this end, the 
following foundational axioms stipulate that function o behaves like set union 
with 0 as the empty setfl 

Zi O (z 2 O Zs) = (zi O Z 2 ) O Z 3 -•Holds{f, 0) 

Zio Z2 = Z20 Zi Holds{fi,f) D / = /i 

zoz = z Holds{f, zi o Z2) 13 Holds{f,Zi) V Holds{f,Z2) 

zo% = z (V/) {Holds{f, Zi) = Holds{f, Z 2 )) D Zi = Z 2 

(V<P)(3z)(V/) {Holds{f,z)=<P{f)) 

where ^ is a second-order predicate variable of sort fluent and the macro 
Holds means that a fluent is part of a state: 

Holds {f, z) = (3z') z = f o z' (1) 



^ Free variables in formulas are assumed universally quantified. Variables of sorts 
ACTION, SIT, fluent, and STATE shall be denoted by the letters a, s, /, and 
2 , resp. The function o is written in infix notation. 
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The very last one of the axioms above stipulates the existence of a state for all 
possible combinations of fluents. A second macro, which reduces to 0, is used 
for fluents holding in situations: 



Holds{f,s) = Holds{f, State{s)) 



As an example, consider a blocks world axiomatization using the fluent 
functions On{x,y), GluedToTable{x), and Has{r,x) denoting, resp., whether 
block a: is on 2 / (which could either be another block or the constant Table), 
whether block x is glued to the table, and whether robot r is in possession 
of X. Suppose, that in the initial state it is known that blocks A and C are on 
the table and B is on C; that no block y is on top of block A or B; and that 
robot Robbie is in possession of glue: 

Holds{On{A, Table), Sq) A Holds{On{C , Table), Sq) A Holds{On{B , C), Sg) 

A (Wy) {-'Holds{On{y, A), Sg) A ~'Holds{On{y, B), So)) (2) 

A Holds{Has{Robbie, Glue),Sg) 

Assuming uniqueness of names for all functions with range fluent, the macro 
definitions and the foundational axioms imply that Q is equivalent to, 

{3z) ( State(Sg) = On{A, Table) o On{C, Table) 

o On{B, C) o Has{Robbie, Glue) o z 
A (Wy) {-'Holds{On{y, A), z) A ~'Holds{On{y, B), z)) (3) 

A -'Holds {On{A, Table), z) A^Holds{On(C, Table), z) 

A -'Holds {On{B , C), z) A -'Holds {Has {Robbie, Glue), z) ) 

The reader may notice that the constraints on sub-STATE z not only reflect the 
negated statements in m but also the fact that the fluents On{A, Table) etc. 
do not recur. This will allow to quickly infer the result of removing any of these 
fluents from State{Sg) as a negative effect. 

The Frame Problem is solved in the Fluent Calculus using so-called state 
update axioms, which specify the difference between the states before and after 
an action. The axiomatic characterization of negative effects, i.e., facts that 
become false, is given by this inductive abbreviation, which generalizes STRIPS- 
style update to incomplete states: 

z' = z — f = [z' o f = z V z' = z] A -'Holds{f, z') 
z' = z - {fio ...o f^o /„+i) = 

{3z") {z" = 0 - (/i o . . . o /„) A z' = z" - /„+i) 

On this basis, the following is the general form of a state update axiom for a 
(possibly nondeterministic) action A{ x ) with a bounded number of (possibly 
conditional) effects: 
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Poss{A{x), s) D (3t/i) (Z\i A State{Do{A{x), s)) = {State{s) o ■(?+) — ) 

V ... V (4) 

(3y„) {An A State{Do{A{x), s)) = {State{s) o r?+) — ’d~) 

where the sub-formulas Ai{x , y i, State{s)) specify the conditions on State{s) 
under which A{ x ) has the positive and negative effects and "d” , resp. 
Both and are state terms composed of fluents with variables among 
X, VzU 

Consider, e.g., the ACTION terms Move{r,u,v,w) and GlueToTable{r,x), 
denoting the action of robot r moving block u away from v onto tc, and gluing 
block X to the table, resp. The direct effects of these two actions can be defined 
by these state update axioms: 

Poss{Move{r, u, v, w),s)d 

State{Do{Move{r,u,v,w), s)) = {State{s) o On{u,w)) — On{u,v) , . 
Poss{GlueToTable{r, x), s) Z) 

State{Do{GlueToTable{r,x), s)) = State{s) o GluedTo Table {x) 

Put in words, after moving u it is on w and no longer on v, and after 
gluing X this block is glued to the table. Recall specification ®, and sup- 
pose, for the sake of argument, that Poss{Move{Robbie, A, Table, B), So) . Let 
= Do{Move{Robbie, A, Table , B) , Sq) ■ Then the state update axiom for Move 
in 0 implies 



State(Si) = {State(So) o On{A,B)) — On{A, Table) 

Replacing State{So) by an equal term according to @ yields, after applying 
the macro for negative effects and performing simplification, 

(3z) State{Si) = On{C, Table) o On{B, C) o Has{Robbie, Glue) o zo On{A, B) 

We have now obtained from an incomplete initial specification a still partial 
description of the successor state, which in particular includes the unaffected 
fluents On{C, Table), On{B,C), and Has {Robbie, Glue). These fluents have 
thus survived the computation of the effect of the action and so need not be 
carried over by separate axioms now. Moreover, knowledge specified in (0 as 
to which fluents do not hold in z applies to the new state, which includes z, 
just as well. Thus, all unchanged fluent values have been concluded to persist 
without applying extra inference steps. 



2.2 State Update Axioms with Ramifications 

In the Fluent Calculus for ramifications, indirect effects are inferred by the suc- 
cessive application of so-called causal relationships, which state under what con- 
ditions an effect triggers another one P3- A causal relationship is formally spec- 
ified with the help of the expression Gauses{e, g, z, s) where e (the triggering 

^ If the conditions Ai are not mutually exclusive, then the action is nondeterministic. 



294 



Y. Martin and M. Thielscher 



effect) and g (the ramification, i.e., indirect effect) are possibly negated atomic 
fluent formulas and z is a state and s a situation. The intuitive meaning is 
that the change to e causes the change to g in state z and situation s. 

For example, let Ab{Movable{x), Glued) be a new fluent, denoting an 
abnormality wrt. block x being movable due to the fact that x is glued to the 
tabled The following state constraint relates this fluent in the obvious way to 
the fluent GluedToTable(x): 

Holds {Ab{Movable{x), Glued), s) = Holds{GluedToTable{x), s) (6) 

Two accompanying causal relationships specify the causal dependence that a 
block X will become immovable if it gets glued to the table, and that this 
abnormal qualification will disappear if the block gets freed somehow: 

Gauses{GluedToTable{x),Ab{Movable{x), Glued), z, s) , . 

Gauses\-^GluedToTable{x),^Ab{Movable{x), Glued), z,s) 

On the basis of causal relationships, the Ramification Problem is solved by 
causally propagating indirect effects: Starting from the direct effects of an action, 
causal relationships are applied successively. The overall result of performing the 
action is then a fixpoint of such a chain of indirect effects. Formally, in state 
update axioms for ramifications the simple equations State{Do(A{x), s)) = 
{State{s) o df) — as in 0) are replaced by sub-formulas of this form: 

z = (State{s) o "df) — "dif O Ramify {z, t?)'’, -d”, Do(A{x), s)) 

where Ramify {z , e~^ , e~ , s) means that State{s) is a fixpoint of iteratively ap- 
plying causal relationships to state z and effects e+,e“ in situation s. (We 
refer to HU for the formal definition of Ramify by a second-order axiom.) 



2.3 Qualifications in the Fluent Calculus 

The theoretical account of the Qualification Problem uses the binary function 
Ab(a;, y) whose range is the sort fluent. The first argument, x, denotes proper- 
ties like Movable{u) or FMnctzonmg(Grzpper-o/(r)). The second argument, y, 
indicates the cause for the abnormality. For convenience, we use the macros 
Ab(a;,z) and Ab(x,s) to represent that for some y, Ab{x,y) holds in state z 
and situation s, respectively: 

Ab(a:, z) (3y) Holds{Ab{x,y), z) Ab(a::, s) Ab{x, State (s)) 

Instances of the generic ‘abnormality’ fluent are used to summarize the abnormal 
qualifications of actions, that is, obstacles which are a priori unlikely to happen 
and therefore need to be assumed away by default in order to jump to the 
conclusion that the action is possible under normal circumstances. E.g., in the 

^ The special fluent Ab will play a key role in our account for the Qualification 
Problem later in this paper. 
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light of abnormal qualifications, the preconditions for our example actions are 
specified by, 

Poss{Move(r, u, v, w),s) = 

Uy^wAv^wA Holds(On(u, v), s) 

A (yy) {-'Holds{On{y,u), s) A ~'Holds{On{y,w), s)) 
A^Ab{Movable{u), s) A Ab{Functioning {Gripper- of (r)), s) 



Poss{GlueToTable{r, x),s) = 

Holds{Has{r, Glue), s)AHolds{On{x, Table), s) A(Vj/) ~'Holds{On{y, x), s) (9) 
A ~^Ab{Movable{x) , s) A ^Ab{ Functioning {Gripper- of {r)) , s) 



As illustrated in the previous section, abnormal qualifications that have been 
caused by the agent himself are accounted for by suitable causal relationships, 
which is how the general problem of anomalous models is overcome; see EH 
To account for abnormal qualifications other than those caused by the agent, 
instances of Ab are allowed to become true during any situation transition as 
a side effect of the mere fact that the very transition takes place. So doing re- 
quires additional causal relationships, which, as opposed to those shown in o, 
describe exogenously caused abnormalities. These are modeled using the predi- 
cates ExogGaused{f , s) and i?a;o 5 C/ncaitsed(/, s), indicating that in situation s 
fluent / arises (resp. vanishes) due to an exogenous cause. The effect of exoge- 
nous causes is specified by corresponding causal relationships. 

Up to this point the treatment of qualifications did not affect the monotonic- 
ity of the solution to the Frame and Ramification Problem. A nonmonotonic 
component, however, is required to minimize abnormal qualifications whenever 
they are not caused by an action that has been performed. This is achieved 
by adding appropriate default rules in the sense of uni, by which the Fluent 
Calculus gets embedded into a default theory. Formally, exogenous influence on 
abnormalities is minimized by default rules of the following form: 

: -<ExogCaused{Ab{x, Exog), s) : ~<ExogUncaused{Ab{x,Exog),s) 
-<ExogCaused{Ab{x, Exog), s) ~<ExogUncaused{Ab{x, Exog), s) 



An accompanying default assumption concerns abnormalities of any kind in the 
initial situation. Their minimization is carried out by defaults of the following 
form: 

: ^Holds{Ab{x,y),So) 

—'Holds{Ab{x, y), Sg) 

If, e.g., the observations suggest no abnormalities initially, then the under- 
lying default theory has a unique extension (in the sense of ESI), which in- 
cludes (Vx) ^Ab(a;, 5'o). (Recall that ^Ab(x, s) means ~'Holds{Ab{x,y),s) for 
any y.) Hence, Q implies that Move{Robbie, A, Table, B) is possible in S'g 
given initial state (|2I). If this action nonetheless fails, that is, if the observation 
-•Poss{Move{Robbie, A, Table, B), So) is added, then the default theory admits 
different extensions. These are obtained by applying all defaults except for one 
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instance of (HU with either {x / Movahle{A)'\ or {x/ Functioning {Gripper-of{ 
Robbie))}. As will be shown in Section fUl below, this approach can also be used 
to account for so-called weak qualifications, that is, unexpected effects of ac- 
tions. Furthermore, by appealing to prioritized default logic (Q, one can specify 
qualitative knowledge of the relative likelihood of the various explanations for 
abnormal qualifications (cf. Section tf.4l below) . The accompanying concept of 
preferred extensions helps selecting the most reasonable explanations in case of 
unexpected action. 

3 Addressing the Qualification Problem in FLUX 

3.1 Basic FLUX 

Our system is implemented in the Eclipse-Prolog system with constraints. It is 
an extension of the programming language Fluent Calculus Executor (Flux), 
a recent implementation of the Fluent Calculus based on Constraint Logic Pro- 
gramming m- The distinguishing feature of Flux is to support incomplete 
states, which are modeled by open lists of the form 

ZO = [FI, . . . ,Fm I Z] 

(encoding the state description ZO = FI o . . . o Fm o Z ), along with constraints 
not_holds(F, Z) 

not_holds_all([Xl, . . . ,Xk] , F, Z) 

encoding, resp., the negative statements {3y)~'Holds{F,Z) (where y are the 
variables occurring in F) and (3 y )(VX1, . . . , Xk) ->LfoZds(F, Z) (where y are 
the variables occurring in F except XI, . . . , Xk ). These two constraints are used 
to bypass the problem of “negation as failure” for incomplete states. In order 
to process these constraints, so-called declarative Constraint Handling Rules ^ 
have been defined and proved correct under the foundational axioms of the Flu- 
ent Calculus. In addition, the core of Flux contains definitions for holds (F,Z) , 
by which is encoded macro and update (Zl,ThetaP,ThetaN,Z2) , which 
encodes the state equation Z2 = (Z1 o ThetaP) — ThetaN . The following is an 
encoding in Flux of the precondition (0 and state update axiom 0 with 
ramifications of the action Move , 

poss (move (R, U, V, W) , Z) 

is_robot(R), is_block(U) , is_block(V) , is_block(W) , U\=W, V\=W, 
not_holds_all(Y, on(Y, U) , Z) , holds(on(U, V), Z) , 
not_holds_all(Y, on(Y, W) , Z) , 
not_holds_all(Y, ab(mov(U), Y) , Z) , 
not_holds_all(Y, ab(func(grip(R)) , Y) , Z) . 

state_update (Z1 , move(R,U,V,W) , Z2, S, H) 
update(Zl, [on(U,W)] , [on(U,V)] , Z3) , 
ramify(Z3, [on(U,W)] , [on(U,V)] , Z2, S, H)). 

where the variable H stands for the history list as defined below. 



Addressing the Qualification Problem in FLUX 297 



3.2 Overview 

Our system implements the defaults of the underlying default theory without the 
need of any special theorem prover. Rather, a modified version of the planning 
algorithm together with the internal Eclipse-Prolog inference mechanisms is used 
to construct the extensions of the underlying default theory, which entail the 
possible explanations for unexpected action failures. This also means that if a 
plan can be executed without any exceptions to the normal execution of actions 
then no additional computations are needed. In this case, the implementation will 
behave like any other system without an approach to the Qualification Problem. 

In this work, we have used a search process with an associated level in or- 
der to find the most likely explanations first. In terms of the underlying default 
theory (with priorities among defaults) this means to search for the least pre- 
ferred default that does not apply. The system also replaces previously consid- 
ered explanations in case a default is no longer preferred in context of a formerly 
established explanation. Of course, the replacement is only performed if it is 
consistent with the executed action sequence. 

In the following we sketch the course of the program and give references to 
the next subsections: 

1. Definition of the Task 

1.1 Define the initial state. This operation includes the verification that 
the state is consistent wrt. the state constraints. 

1.2 Define the goal state. 

2. The Planning Algorithm 

2.1 Find a plan. State updates are computed with ramifications (cf. Section 
E3). An agent has no influence on exogenously caused abnormalities and 
cannot yet have caused any abnormal qualification in the initial situation. 
Therefore, all defaults on the absence of exogenous causes are assumed 
to apply during the planning process. Iterative deepening is applied as 
search strategy. Furthermore, the program uses some heuristics to cut 
down the search space. 

2.2 Double-check the computed plan. Planning with incomplete state 
information requires to verify an established plan against both, not 
achieving the goal and not being executable. 

3. Plan Execution 

3.1 Execute the computed plan step by step. The execution of each 
action is monitored. If no action fails and all the actions achieve their 
intended effects then the goal state will be reached and the program 
terminates. Otherwise, the program proceeds with step 4. 

4. Explanation of and Recovery from Action Failures 

4.1 Search for an explanation for the unexpected observation. If 
an action surprisingly fails or does not produce all the intended effects 
then an abnormal qualification must have occurred. This means that 
at least one default of the underlying default theory can no longer be 
applied. For the intended applications of the program it is in most cases 
sufficient to search for atomic explanations, i.e., where the application 



298 



Y. Martin and M. Thielscher 



of exactly one default is blocked during the construction of each of the 
possible extensions. Furthermore, we assume that there is always at most 
one explanation at each level. Using the built-in inference mechanisms 
of the Eclipse-Prolog system, each extension of the underlying default 
theory is considered where one instance of a default rule is blocked. If 
the non-application of such a default rule entails the observation then an 
explanation has been found and the search process stops (cf. Sectionl^j) . 

4.2 Find the explanation with the highest priority first. Using a 
search process with levels, the search will find only explanations with 
a priority higher or equal to the current level of the search algorithm 
(cf. Section ld.4t . Only if there is no such very likely explanation that 
accounts for the observation then the program searches with the next 
lower level and thus considers less likely explanations. 

4.3 Determine the current state and replan. The search process of the 
program can deliver a strong or a weak qualification (cf. Section Id. dp as 
an explanation. In both cases, this new information is integrated into 
the current state, which becomes the initial state of the new planning 
problem. Then the planning algorithm is used to find a new plan despite 
the encountered abnormality. Hence, the program proceeds with step 2. 

An important concept in the approach to the Qualification Problem in Flux is 
the notion of a history list. Such a list contains all the abnormalities that have 
occurred so far during the execution of the program. Each entry in the list has 
three parts. The first part describes the abnormality predicate with the prop- 
erty and the cause. In the second part the situation, in which the abnormality 
occurred, is stated. The third part denotes the possible observation which lead 
to the occurrence of the abnormality. In addition to these entries, the history 
list contains the current level of the search process. The history list is modeled 
by an open list with a tail variable. 

3.3 Constructions of Extensions 

In this section we show how extensions of the underlying default theory are 
constructed for strong qualifications of actions. The inference process is similar 
for weak qualifications. 

Effects with an exogenous cause are implemented by causal relationships of 
the following form: 

causes(_, ab(mov(X), exog) , Z, S, H) block(X) , 

X\= table, exogcaused(ab(mov(X) , exog), S, H) . 

causes(_, -(ab(mov(X) , exog)), Z, S, H) block(X) , 

X\=table, exoguncaused(-(ab(mov(X) , exog)), S, H) . 

Please note that the indirect effects Ab{Movable{x), Exog) in these clauses are 
not conditioned on any direct effect. Consequently, these positive or negative 
indirect effects occur as a side effect of every transition whenever the predicates 
ExogCaused{Ab{x, Exog), s,h) or ExogUncaused{Ab(x, Exog), s,h) hold, where 
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the variables s and h stand for the considered situation and history list, re- 
spectively. Indirect effects wrt. the abnormality Ah(Functioning {Gripper-of (r)) , 
Exog) are similarly encoded. 

Predicate ExogCaused and predicate ExogUncaused are specified in a sim- 
ilar way in our program. For brevity we present only the important details of 
the definition for ExogCaused . 

exogcausedCAB, S, H) ... 

top(H, H2), 
lengthl(H2,0) , 
holds(h(AB, S), H) ) ) ) . 

The clause uses the auxiliary predicates Top{h\, h 2 ) and Lengthl(h,n) . The 
predicate Top takes the present history list and yields the currently considered 
abnormality predicate, if any. The predicate Lengthl delivers the length of a list. 
The definition of the clause ensures that exactly one abnormality is considered 
as an explanation at any stage of the search process. 

Extensions are constructed during state updates with ramifications. For each 
action all possible extensions of the underlying default theory are tried until the 
established extension accounts for the observation. This is achieved by block- 
ing the application of each default one after the other. The ExogCaused or the 
ExogUncaused predicate is assumed to hold and the specific default is blocked 
by means of adding the corresponding abnormality as indirect effect to the cur- 
rent state. The addition is performed by the clauses for causal relationships 
together with the clauses for state update axioms with ramifications. The pred- 
icates ExogCaused and ExogUncaused ensure that only one default is blocked 
at a time, and the Eclipse-Prolog SLDNF-resolution with backtracking yields all 
the extensions. The possible defaults for the initial situation are computed in the 
same way. To this end, the special constant “e” (read “no-op”) is introduced. 
It denotes the empty action without any positive or negative direct effects. This 
action is always possible and is only executed as the very first action. 

As illustration, consider the initial state as specified in (0 together with the 
following definitions for blocks and robots: 

is_robot (robbie) . is_block(table) . is_block(a) . 

is_block(b) . is_block(c) . 

The query Init{zo, h), Res{e, zq, 5o, zi, si, h ) , \+Poss{Move{Robbie, A, Table, 
B),zi) admits two computed answer substitutions, where the predicate Init 
denotes the initial state and the predicate Res denotes the execution of exactly 
one action: 

{zi/[Ah{Movable{A), Exog), On{A, Table), On{B,C), On{C, Table), 
Has{Robbie, Glue) \z\, h/[H{Ah{Movable{A), Exog), Sq) |hi]} 

i zi/\ Ab(Eunctioning( Gripper-of (Robbie)), Exoq), On(A, Table), On(B,C), 
On(C, Table), Has (Robbie, Glue) \ z], 
h/[H (Ab(Functioning (Gripper- of (Robbie)) , Exog), Sq) \ hi]} 
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These substitutions are computed by applying all defaults except for one instance 
of (II 111 with {x / Movable {A)} for the first substitution and {x / Functioning { 
Gripper- of {Robbie))} for the second one. This way, our program determines the 
two extensions in this example as described at the end of Section ITT?I 

Extensions are constructed using the inference mechanism of the Eclipse- 
Prolog system, i.e., an implementation of the SLDNF-resolution. This resolution 
scheme is sound. For the soundness proof it should be referred to Lloyd HD!. 

In our implementation extensions are constructed using the 9-ary predicate 
RunDefault . We present a schematic version of its encoding: 

rundefaultC [] , Z, Z, S, S, SW, HI, H2, R) 

\+ poss (R, Z) , ... 

rundefaultC [FiL] , ZO, Z, S, SF, SW, HI, H2, R) 
append! [A] , [E] , F) , ! , ... 
res(A, ZO, S, Zl, SI, HI) , ... 

currentCSl, HI, HH) , (HH= [] ; HH=[h(_,_,o(M,_))] , \+poss(M, Zl)), 

rundefaultCL, Z2, Z, SI, SF, SW, HI, H2, R) . 

The predicate RunDefault has a recursive definition. The recursion is performed 
on its first argument. This argument represents the executed sequence of actions 
as a list. The last argument of RunDefault stores the current observation. It is 
the action that failed unexpectedly for the reason of a strong abnormal qualifica- 
tion. Thus, for the predicate to terminate successfully the computed explanation 
must account for the observation after having performed the complete sequence 
of actions. Of course, all other observations recorded in the history list must also 
be taken into account during the search. To this end, the auxiliary predicate 
Current{s, hi, / 12 ) is used. It checks for the current situation of the search pro- 
cess, whether an unexpected action failure has occurred in this situation during 
the execution of the plan. If this is the case then the predicate Current delivers 
the corresponding action. Otherwise, the empty list is returned. 

3.4 Selection of the Preferred Extension 

Extensions are constructed in accordance with the underlying set-prioritized 
default logic, which is an extension of the prioritized default logic and can be 
used to define preferences between defaults HE]- The preference relations in 
the program are defined with or without context-dependency. The first case 
defines a set-preference ordering among defaults so that any two instances of 
a default concerning the same object are compared to an instance of another 
default concerning the object. If there is no context of a previous explanation 
then the second case gives a definition of general preference between defaults. 
As illustration, consider the implementation for the running example together 
with the following preference relations: 

level([12,ll,10,9]) . 
preference(ab(mov(_) , _) , nc, 9). 
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preference (abCfunc (grip (_)) , _) , nc, 10). 
preference (ab(mov(_) , _) , ab(func(grip(_)) , _) , 11). 

That is, without context the explanation Ah{Functioning(Gripper-of{x)),y) is 
more likely than the explanation Ah(Movable{x),y) . In contrast, the explana- 
tion Ah{Movable{x),y) is preferred over the assumption of two explanations of 
the form Ab(Functioning{Gripper-of{x)),y) . 

The preference relations are taken into account when using the predicates 
ExogGaused and ExogUncaused . The following shows the part of the clause 
without context information for the predicate ExogGaused : 

exogcaused(AB, S, H) ... 

preference (AB , nc, P) , 
member (1(D) , H) , !, P>=D, 
top(H, H2) , ... 

The priority of the currently considered explanation is obtained. Afterwards, 
the current level of the search is obtained from the history list. Further on, the 
computed priority of the considered explanation is compared to this level. If the 
priority is at least as high as the current level then the procedure continues as 
described in Section roi 

The change to the next lower level in the search is encoded as: 

..., level (LEVEL) , member (LE, LEVEL), changed (LE, H, H3) , 
rundefault(L2, ZN, ZF, sO, SV, SW, H3, HF, W), ... 

The auxiliary predicate GhangeD sets the current level in the history list. If the 
predicate RunDefault fails then the Eclipse-Prolog system backtracks and the 
predicate Member chooses the next lower level for the search process. 

For example, consider the query Init{zo,h), Level{level), Memberile, level), 
GhangeD{le,h,hi), Res{e, Zq, Sq, Zi, Si,hi), \+Poss{Move{Robbie, A, Table, B), 
Zi) , where the initial state is specified as in 0 and the preferences are de- 
fined as above. All preferred extensions of the underlying default theory entail 
Ab(Eunctioning{Gripper-of (Robbie)) , Sq) . In accordance with the preferred ex- 
tension the query yields the computed answer substitution: 

{zi / [Ab(Eunctioning (Gripper-of (Robbie)) , Exog), On(A, Table), 

On(B, C), On(C, Table), Flas(Robbie, Glue) \ z], 
h/[le(lQ), F[ (Ab(Eunctioning (Gripper- of (Robbie)) , Exog), So) \ hi]} 



3.5 Weak Qualifications 

Weak qualifications, that is, failure to produce expected effects, are denoted and 
minimized in the same way as strong qualifications. Causal relationships and 
preference relations for weak qualifications are implemented as illustrated by 
the following clauses: 
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causes(_, ab(trans(X), exog) , Z, S, H) block(X) , 

X\= table, exogcaused(ab(trans(X) , exog), S, H) . 

preference (ab (trcLns (_) , _) , nc , 11). 

The weak qualification Ah(Transportable{x),y) means that block x is slip- 
pery and will slip out of the gripper when transported over long distances. The 
construction of extension for default theories, which contain defaults with ab- 
normality predicates regarding weak qualifications of actions, is performed in a 
similar fashion as described in Section I, 3. ,31 The information given in Section 1,3.41 
regarding the preferred extension also holds for weak qualifications. 

Fluents denoting weak qualifications strengthen the antecedents of state up- 
date axioms. This is in contrast to strong qualifications where abnormality pred- 
icates occur in action precondition axioms. Thus, a suitable state update axiom 
with a possible weak qualification for the action Move{r, u, v, w) is encoded as: 

state_update (Z1 , move(R,U,V,W) , Z2, S, H) 

(not_holds_all(Y, ab(trEins(U) , Y) , Zl) , !, 
updateCZl, [on(U,W)] , [on(U,V)] , Z3) , 
ramify(Z3, [on(U,W)] , [on(U,V)] , Z2, S, H)); 

(holds (ab (trans (U) , Y) , Zl) , 

(V\=table , 

update(Zl, [on(U, table)] , [on(U,V)] , Z3) , 
rcmiify(Z3, [on(U, table)] , [on(U,V)] , Z2, S, H) ; 

V==table, ramify(Zl, [] , [] , Z2, S, H))). 

For this state update axiom let us consider the query Init{zQ, h), Res{e, zq, Sq, 
Zl, si, h), Res{Move{Rohbie, B, C, A),z\, s\,Z 2 , S 2 , h), NotHolds{On{B , A), Z 2 ) , 
where the initial state is specified as in O- This query yields the computed 
answer substitution: 

{z 2 /[On{B, Table) , Ah{Transportable{B) , Exog) , On{A, Table), On[C, Table), 
Has{Robbie, Glue) \z], h/[H{Ah{Transportable{B), Exog), Sq) \ hi]} 

Thus, the program concluded that the weak qualification Ah (Transportable (B) , 
Exog) occurred, and that block B can be found somewhere on the table. 

4 Experiments 

Our program has been tested with a LEGO@ MINDSTORM"'"'^ robot in 
a delivery scenario. The main component of such a robot is a programmable 
brick. It is referred to as RCX (Robotic Command Explorer) and has as its core 
a Hitachi H8 microcontroller. Our robot has a light sensor and a pushbutton 
sensor and two motors attached to the input ports and output ports of the 
RCX, respectively. An infrared port is used for the communication between the 
computer and the RCX while a user program is running on the RCX. Such 
programs only realize simple behaviours like following a line. 
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All high level control is performed by the Eclipse-Prolog system. This in- 
cludes the planning process and the monitoring of the executions of actions by 
means of exogenous events. In case of an unexpected action failure, the system 
searches for explanations. The robot only executes primitive actions. Addition- 
ally, it observes the occurrence of exogenous events and reports them to the 
Eclipse-Prolog system. The communication between the system and the robot is 
achieved through message exchange using a module from the Legolog system |0| . 

The robot is supposed to deliver objects from one office to another, where a 
cardboard with bright markers as offices and black lines as tracks denotes the 
floor plan. The robot has solved the task in our example scenario if there are 
no more requests in the current state and the robot has returned to its initial 
position. Two kinds of abnormalities with an appropriate preference relations 
between them have been defined for this scenario. 

For this example domain our program was able to find explanations for un- 
expected action failures and to recover from them. In the scenario the system 
concluded that the robot had missed a marker long before the observance of 
an exception. In the search process all previously executed actions and related 
observations were taken into consideration to generate the most likely explana- 
tion first. With the established explanations the program was able to infer the 
robot’s current position and to find a new plan to solve the task. 

5 Summary 

We have presented an extension of the action programming language Flux which 
copes with the Qualification Problem. Our approach builds on the theoretical 
work of where the Fluent Calculus has been embedded into a default theory 
to account for abnormal qualifications of actions. Our system allows to generate 
plans under the assumption that actions succeed as they normally do, and to 
reason about these assumptions in order to recover from unexpected action fail- 
ures. Furthermore, it supports the specification of preferences among the default 
assumptions, by which is aided the search for reasonable explanations in case of 
unexpected action failure. While action programming languages have been ex- 
tended by execution monitoring in the past, e.g., our system is the first which 
is based on a formal approach to the Qualification Problem. It thus provides a 
declarative approach to troubleshooting. The crucial advantage of our approach 
is that explaining unexpected action failures is carried out on the basis of the 
same action specifications and reasoning techniques which are used for planning. 
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Abstract. The paper presents a new technique for extracting symbolic ground 
facts out of the sensor data stream in autonomous robots for use under hybrid 
control architectures, which comprise a behavior-based and a deliberative part. 
The sensor data are used in the form of time series curves of behavior activation 
values. Recurring patterns in individual behavior activation curves are aggregated 
to well-defined patterns, like edges and levels, called qualitative activations. Sets 
of qualitative activations for different behaviors occurring in the same interval 
of time are summed to activation gestalts. Sequences of activation gestalts are 
used for defining chronicles, the recognition of which establishes evidence for the 
validity of ground facts. The approach in general is described, and examples for 
a particular behavior-based robot control framework in simulation are presented 
and discussed. 



1 Background and Overview 

There are several good reasons to include a behavior-based component in the control 
of an autonomous mobile robot. There are equally good reasons to include in addition 
a deliberative component. Having components of both types results in a hybrid control 
architecture, intertwining the behavior-based and the deliberative processes that go on 
in parallel. Together, they allow the robot to react to the dynamics and unpredictability 
of its environment without forgetting the high-level goals to accomplish. Arkin IArk981 
Ch. 6] presents a detailed argument and surveys hybrid control architectures; the many 
working autonomous robots that use hybrid architectures include the Remote Agent 
Project IMNPW9H RemOOl as their highest-flying example. 

While hybrid, layered co ntrol architectures for autonomous robots, such as Saphira 
IKMSR97I or 3T |IRFG~*~97| are state of the art, some problems remain that make it a 
still complicated task to build a control system for a concrete robot to work on a concrete 
task. To quote Arkin HArk98l p. 207], 

the nature of the boundary between deliberation and reactive execution is not 
well understood at this time, leading to somewhat arbitrary architectural deci- 
sions. 
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One of the problems is to keep up-to-date the symbolic world representation for 
the deliberative component. There are solutions to important parts of that problem, 
such as methods and algorithms for sensor-based localization to reason about future 
navigation actions: (FBT99fl presents one of the many examples for on-line robot pose 
determination based on laser scans. If the purpose of deliberation is supposed to be more 
general than navigation, such as action planning or reasoning about action, then the 
need arises to sense more generally the recent relevant part of the world state and update 
its symbolic representation based on these sensor data. We call this representation the 
current situation. 

The naive version of the update problem “Tell me all that is currently true about 
the world!" needs not be solved, luckily, if the goal is to build a concrete robot to 
work on a concrete task. Only those facts need updating that, according to the symbolic 
domain model used for deliberation, are relevant for the robot to work on its task. Then, 
every robot has its sensor horizon, i.e., a border in space and time limiting its sensor 
range. The term sensor is understood in a broad sense: It includes technical sensors 
like laser scanners, ultra sound transducers, or cameras; but if, for example, the arena 
of a delivery robot includes access to the control of an elevator, then a status request 
by wireless Ethernet to determine the current location of the elevator cabin is a sensor 
action, and the elevator status is permanently within the sensor horizon. We assume: The 
world state information within the sensor horizon is sufficient to achieve satisfying robot 
performance. 

This said, the task of keeping the facts of a situation up-to-date remains to continually 
compute from recent sensor data and the previous situation a new version of the situation 
as far as it lies within the sensor horizon. The computation is based on plain, current 
sensor values as well as histories of situations and sensor readings or aggregates thereof. 
Practically, we cannot expect to get accurate situation updates instantly; all we can do 
is make the situation update as recent, comprehensive, and accurate as possible. 

This paper contributes to this task an approach of using histories of activation values 
in behavior-based robot control systems (BBSs) |[Mat99l as a main source of information 
for situation update. This approach is useful for three reasons: 

- Activation values are calculated anyway in most BBSs to allow for arbitration or 
merging between behaviors; they can be used at no additional computation cost, 
provided that they are sufficiently fine grained. 

- An activation value is grounded in sensor readings and, by definition, evaluates them 
in a way tailored to its respective behavior; using activation values like aggregated 
sensor readings yields automatically an action-centered way of “looking through 
the sensors". 

- To have a practical hybrid robot control, the symbolic world model must be in accord 
with the inventory of behaviors anyway; using activation value histories in situation 
update only makes even more explicit the need to co-design the BBS and deliberative 
control components. 

Our approach is not in principle limited to a particular combination of deliberation 
component and BBS, as long as the BBS is expressed as a dynamical system and in- 
volves a looping computation of activation values for the behaviors. Some additional 
requirements apply that will be clarified in the paper. 
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All demo examples are formulated in a concrete BBS framework, namely, Dual 
Dynamics (DD, IJC97I ). which has also inspired our abstract view of BBSs. Our view 
of building hybrid robot controllers involving a BBS as reactive component is shaped by 
our work in progress on the DD&P robot control architecture HHJZM98 ;HS01I . which 
blends DD controllers with action plans generated by a classical propositional planner 
(concretely, IPP 1KNHD971 ) as the central deliberation component. Mind, however: 
The method for fact extraction from activation value histories of BBSs presented here 
is potentially applicable in BBS frameworks other than DD, as will be discussed at the 
end of the paper. 

The rest of this paper is organized as follows. In Sec.Q we present our approach of 
formulating BBSs as dynamical systems and give a detailed example in Sec. El To provide 
some background concerning complete robot control systems, we then (Sec.® sketch 
how we assume the deliberation component interferes the BBS control component. Sec. El 
contains the technical contribution of the paper, describing in general as well as by way 
of example the technique of extracting facts from BBS activation value histories. Sec. El 
discusses the approach and relates it to the literature. Sec.Clconcludes. 

2 BBSs as Dynamical Systems 

We assume a BBS consists of two kinds of behaviors: low-level behaviors (LLBs), which 
are directly connected to the robot actuators, and higher-level behaviors (HLBs), which 
are connected to LLBs and/or HLBs. Each LLB implements two distinct functions: a 
target function and an activation function. The target function for the behavior b provides 
the reference tb for the robot actuators {"what to do") as follows: 



where ft is a nonlinear vector function with one component for each actuator variable, 

is the vector of all inputs from sensors, s J is the vector of the sensor-filters and 
is the vector of activation values of the LLBs. By sensor-hlters - sometimes called virtual 
sensors - we mean markovian and non-markovian functions used for processing specific 
information from sensors. 

The LLB activation function modulates the output of the target function. It provides 
a value between 1 and 0, meaning that the behavior fully influences, does not influence 
or influences to some degree the robot actuators. It describes when to activate a behavior. 
For LLB b the activation value is computed from the following differential equation: 



Eq. El gives the variation of the activation value ab^LLB of this LLB. gb is a nonlinear 
function. OCTb allows the planner to influence the activation values, see Sec.0 The 
scalar variables OnFb and OffFb are computed as follows: 



tb — fb{s^ ,S^ ,0 :J^lb) 



( 1 ) 



ab,LLB = gb{o!b, LLB, OnFb, OffFb, OCTb) 



(2) 



OnFb — Ub{s^ , ,oi^lb,c^hlb) 
OffFb = Vb{s'^ , sj , 



( 3 ) 



(4) 
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where Ub and Vb are nonlinear functions. The variable OnFb sums up all conditions 
which recommend activating the respective behavior (on forces) and OffFb stands for 
contradictory conditions to the respective behavior (off forces). 

The HLBs implement only the activation function. They are allowed to modulate 
only the LLBs or other HLBs on the same or lower level. In our case, the change of 
activation values for the HLBs ab^HLB are computed in the same manner as Eq.|3 

The reason for updating behavior activation in the form of Eq.Qis this. By referring 
to the previous activation value ab, it incorporates a memory of the previous evolution 
which can be overwritten in case of sudden and relevant changes in the environment, 
but normally prevents activation values from exhibiting high-frequency oscillations or 
spikes. At the same time, this form of the activation function provides some low-pass 
filtering capabilities, deleting sensor noise or oscillating sensor readings. 

Independent from that, it helps to develop stable robot controllers if behavior ac- 
tivations have a tendency of moving towards their boundary values, i.e., 0 or 1 in our 
formulation. To achieve that, we have implemented gb in Eq.Elas a bistable ground form 
(like in [IBGG+QQI| for a RoboCup application of a BBS of the same type) providing 
some hysteresis effect. Without further influence, this function pushes activation values 
lower/higher than some threshold /3 (typically [3 = 0.5) softly to 0/1. The activation 
value changes as a result of adding the effects coming from the variables OnF, OffF, 
OCT and the bistable ground form. Exact formulations of the gb function are then just 
technical and unimportant for this paper. 

The relative smoothness of activation values achieved by using differential equations 
and bistability will be helpful later in the technical contribution of this paper (Sec.01, 
when it comes to derive facts from the time series of activation values of the behaviors. 

In our BBS formulation, behavior arbitration is achieved using the activation values. 
As shown in Eqs. Q-El each behavior can interact with (i.e., encourage or inhibit) every 
other behavior on the same or lower level. The model of interaction between behaviors 
is defined by the variables OnF and OffF. 

The output vector or reference vector r of the BBS for the robot actuators is generated 
by summing all LLB outputs by a mixer, as follows: 



Together with the form of the activation values, this way of blending the outputs of LLBs 
avoids discontinuities in the reference values for the single robots actuators, such as 
sudden changes from full speed forward to full speed backward. 

3 An Example 

To illustrate the notation of Sec. Q we give a demonstration problem consisting of the 
task of following a wall with a robot and entering only those doors that are wide enough 
to allow the entrance. Figure[I]gives an overview about the main part of our arena. The 
depicted robot is equipped with a short distance laser-scanner, 4 infrared side-sensors, 

4 front/back bumpers and some dead-reckoning capabilities. 




( 5 ) 



b 
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Fig. 1. The demo arena and the final robot pose in Example 1 . The robot has started its course at 
the lower right comer, below to the round obstacle. 



We used a simulator based on the DDDesigner prototype tool IBreOOlB GG+ 991 . The 
tool allows checking isolated behaviors or the whole BBS in designated environmental 
situations (configurations). 

The control system contains three HLBs and six LLBs, see Fig. El RobotDirection 
and RobotV elocity are the references for the two respective actuators. We have the 
following FILBs (cf. Fig.EJ: 

CloseToDoor is activated if there is evidence for a door; 

InCorridor is active while the robot moves inside a corridor; 

TimeOut was implemented in order to avoid getting stuck in a situation, see Sec.|3 

The LLBs are the following; 

TurnToDoor is activated if the robot is situated on a level with a door; 

GoThruDoor is activated after the behavior TurnToDoor was successful; 
FollowRightWall is active when a right wall is followed; 

FollowLeftWall is active when a left wall is followed; 

AvoidColl is active when there is an obstacle in the front of the robot 
Wander is active when no other LLB is active. 

Most of the implemented behaviors are common for this kind of tasks. However, we 
decided to split the task of passing a door in a sequence of two LLBs. This helps 
structure, maintain and independently improve these two behaviors. 
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Clos^D^or InCorridor TimeOut 



GoThruDoor 



FollowLeftWall 



AvoidColl 

Wander 



Fig. 2. The behavior inventory for our examples as in a screen shot from the DDDesigner tool. Big 
circles denote FlLBs; squares denote LLBs; the hollow icons at the bottom denote robot actuator 
reference values. Arrows denote control flow between LLBs and actuators. The influence structure 
between behavior activations is not shown; the small circles are of no importance here. 



To give a simple example of BBS modeling, here are the “internals” of Wander: 



OnTvVander 
Off Twander 

RobotDirectiony\jander 
RobotV elocitywander 

fwander 

(^Wander 



ki{l — CtCloseToDoor) * (1 — CtpoiiowRightWall) * (6) 

(1 — CtFollowLeftWall) * (1 — CtAvoidColl) 

^20!AvoidColl + fcaCtCloseToDoor + (7) 

^40:FollowRightWall + ^saFollowLeftWall 
randomDirection{) (8) 

mediumSpeed (9) 

RobotDirectiony\jander 
RobotV elocityvjander 



5Wander(ctWanderj OnTwdri 0//Twdrj OCTyjdr) (H) 



where ki . . .k^ are empirically chosen constants. randomDirection{) could he every 
function that generates a direction which results in a randomly chosen trajectory. 

Due to its product form, OnFwander can only he remarkably greater than zero if all 
included ab are approximately zero. 0//Fwander consists of a sum of terms allowing 
every included behavior to deactivate Wander. Both terms are simple and can be calcu- 
lated extremely fast, which is a guideline for most BBSs. The OCT term will be briefly 
explained in the next section. 

Fig.0shows the activation value histories generated during a robot run, which will be 
referred to as Example 1 . The robot starts at the right lower edge of Fig.[I]with Wander in 
control for a very short time, until a wall is perceived. This effect is explained by Eq.|^ 
While the robot starts to follow the wall, it detects the small round obstacle in front. 
In consequence, two LLBs are active simultaneously: AvoidColl and FollowLeftWall. 
Einally, the robot follows the wall, ignores the little gap and enters the door. In the 
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examples for this paper, FollowRightWall is always inactive and therefore not shown in 
the activation value curves. 



0 2 4 6 8 10 12 14 16 18 20 22 24 2 6 




0 2 4 6 8 10 12 14 16 18 20 22 24 2 6 



Fig. 3. The activation value histories for Example 1. The numbers are time unit for reference. 



This exemplifies the purpose of a slow increase in behavior activation. 
FollowLeftWall should only have a strong influence to the overall robot behavior if 
a wall is perceived with both side-sensors for some time, so as to be more sure that 
the robot really has sensed a wall. The small dent in the activation of FollowLeftWall 
(around the time f = 4) is explained by perceiving free space with one side-sensor. If 
both side-sensors detect free space this behavior would be deactivated. The turning to 
the door is described by rising/falling edges of some activation values. The second rise 
of AvoidColl (after t = 22) is caused by the door frame, which pops into sight as a 
close obstacle at the very end of the turning maneuver. Effectively, the collision avoid- 
ance guides the robot through the door. Finally GoThruDoor gets slowly deactivated 
allowing other behaviors to take control of the robot. 

The HLBs CloseToDoor and InCorridor describe global states, thereby modulating 
the interaction, activation and sequencing of the LLBs. 

4 From Plans to BBSs: Blending Behaviors with Operators 

The technical contribution of this paper is an approach to enhancing the information flow 
from the BBS to the deliberation part in hybrid robot control systems. Before coming to 
that in Sectional we want to sketch the control flow in the opposite direction, from the 



312 



F. Schonherr et al. 



planner to the BBS, to make complete the picture of the entire robot control architecture 
that we have in mind. Not in the focus of the present paper, this description just consists 
of stating the basic principle, and we refer to work on the DD&P control architecture 
IHJZM98 HSOIL which elaborates on the approach. 

The basic idea is this: An action planner continually maintains a current action plan, 
based on the current situation and the current set of user-provided or self-generated mis- 
sion goals. Based on the current plan and the current situation, an execution component 
picks one of the operators in the plan as the one currently to be executed. Plan execution 
is done in a plans-as-advice lPol921 fashion: Executing an operator means stimulating 
more or less strongly the behaviors working in favor of the operator, and muting those 
working against its purpose. Which operator stimulates or mutes which behaviors is an 
information that the domain modeler has to provide along with the domain model for 
the deliberative component and the set of behaviors for the BBS. 

Technically, the influence of the current operator is “injected” into the BBS in terms 
of the Operator-Coupling-Terms (OCT) in the activation functions, see Eqs.EI The 
influence of the current ground operator op gets inside every behavior b through the term 
OCTf,, as follows: 



where Z'^ G {0)1} ^nd is a constant, = 1 iff op influences the behavior 
b. models the immediacy or delay of the operator influence on the behavior. 
expresses whether the operator influence is of the stimulating or the muting sort: If 
= 1, then the respective behavior is stimulated, and muted if Z^^ = 0. Z may be a 
boolean function, returning 0 or 1 conditionally. 

To give an example, assume that the domain model for the deliberation component 
includes an operator GO-IN-RM(a;) modeling the action of some office delivery robot 
to go (from wherever it is) to and enter room x. Let the behavior inventory be the one 
specified in Sec.El Here is a selection of s, Z, and c variables of these behaviors and 
how they should be affected by the ground operator GO-IN-RM(A) : 

'^G^ThmDoor^^ ^ operator does influence the behavior; 

''G^ThruD^r^^ some medium value, causing a tendency to influence activation soon 

after the operator is chosen as being active; 

■^G^hmD^r^^ c/!flrFct(CloseTo(A)), i.e., the characteristic function that returns 
1 if CloseTo(A) is currently in the fact base, and 0 else; 

^ operator should affect its activation (namely, muting it); 

'^AvoidCdi''**^^^ ® collision avoidance should not be affected by the operator (note 

the difference between not affecting and actively muting an activation value) 

5 From BBSs to Plans: Extracting Facts from Activation Values 

We now turn to the method how to extract facts from activation value histories. It is 
influenced by previous work on chronicle recognition, such as IGha96l . 

To start, take another look at the activation curves in Fig. 0in Sec.0 Some irregular 
activation time series occur due to the dynamics of the robot/environment interaction. 




( 12 ) 
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such as early in the AvoidColl and Wander behaviors. However, certain patterns re- 
occur for single behaviors within intervals of time, such as a value being more or less 
constantly high or low, and values going up from low to high or vice versa. The idea 
to extract symbolic facts from activation values is to consider characteristic groups or 
gestalts of such qualitative activation features occurring in chronicles over time. 

To make this precise, we dehne, hrst, qualitative activation values (or briefly, qual- 
itative activations) describing these isolated patterns. In this paper, we consider four 
of them, which are sufficient for defining and demonstrating the principle, namely, ris- 
ing/falling edge, high and low, symbolized by predicates ffe, ij-e. Hi, and Lo, respectively. 
In general, there may be more qualitative activations of interest, such as a value staying 
in a medium range over some period of time. For a behavior b and time interval [ti , < 2 ], 
they are defined as 



Hi(6) 

Lo(5) 

tre(6)[ti,t2] 

l|e(5) [fi, <^ 2 ] 



ab[t] > h 


for all t\ < t < t 2 






ab[t] < 1 


for all t\ < t < t 2 






Oib[tl] = 1 


and ab\t 2 \ = h and 




(13) 


ab increases generally monotonically over 


[^1)^2] 




ab[ti] = h 


: and ab[t 2 ] = 1 and 




(14) 


ab decreases generally monotonically over 


[^1)^2] 





for given threshold values 0 <C < landO < I <C 1, where a;, [f] denotes the value of a;, 
at time t. General monotonicity requires another technical definition, which we skip here 
for brevity. The idea is that some degree of noise should be allowed in, e.g., an increasing 
edge, making the increase locally non-monotonic. In the rather benign example activation 
curves in this paper, regular monotonicity suffices. Similarly, it is not always reasonable 
to use the global constants h, I as Hi and Lo thresholds, respectively. It is possible to 
use different threshold constants or thresholding functions for different behaviors. We 
do not go into that here. Then, it makes sense to require a minimum duration for , ^ 2 ] 
to prevent useless mini intervals of Hi and Lo types from being identified. Finally, the 
strict equalities in Fq. sITH andPHl are unrealistic in real robot applications, where two 
real numbers must be compared, which are seldom strictly equal. Equality ±e is the 
solution of choice here. 

The key idea to extract facts from activation histories is to consider patterns of 
qualitative activations of several behaviors that occur within the same interval of time. 
We call these patterns activation gestalts. We express them formally by a time-dependent 
predicate AG over a set Q of qualitative activations of potentially many behaviors. For 
a time interval [t, t'] the truth of AG{Q) [t, t'] is defined as the conjunction of conditions 
on the component qualitative activations q G Q of behaviors b in the following way: 



case g = Hi(&) then Hi(6)[f, f'] 
case g = Lo(&) then Lo{b)[t,t'] 

case g = jje(&) then jje(&)[fi, ^ 2 ] for some [ti, ^ 2 ] C [f, f'], and Hi(&)[f 2 ,f^] 
case g = jj.e(6) then ^ 2 ] for some [fi, ^ 2 ] C [f, f'], and Lo{b)[t 2 ,t'] 
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Note that it is not required that different rising or falling edges in Q start or end syn- 
chronously among each other or at the interval borders of [t, f] — they only must all 
occur somewhere within that interval. 

For example, AG'({f|'e(GoThruDoor), l|e(TurnToDoor), Hi(CloseToDoor)}) is true 
over [20, 24] in the activation histories in Fig.|3 it is also true over [16, 23] (and therefore, 
also over their union [16, 24]), but not over [16, 25], as CloseToDoor has left its Hi band 
by time 25, and possibly the same for GoThruDoor, depending on the concrete value of 
the h threshold. 

A chronicle over some interval of time [Iq , f] is a set of activation gestalts over sub- 
intervals of [fo, t] with a finite set of n linearly ordered internal interval boundary points 
to <ti <■■■ <tn<t.A ground fact is extracted from the activation history of a BBS 
as true (or rather, as evident, see the discussion below) at time t if its defining chronicle 
has been observed over some interval of time ending at t. The defining chronicle must 
be provided by the domain modeler, of course. 

We give as an example the dehning chronicle of the fact In Room that the robot is 
in some room, such as the one left of the wall in Fig.[D lnRoom[f] is extracted if the 
following defining chronicle is true within the interval [to , t] , where the ti are existentially 
quantified: 

AG'({l|e(GoThruDoor)})[f 4 , t] 

A AG({t|'e(GoThruDoor), U.e(TurnToDoor), Hi(CloseToDoor)})[f 3 , f 4 ] 

A AG({Hi(TurnToDoor, Lo(lnCorridor)})[f2,f3] 

A AG({t|'e(TurnToDoor),'f|'e(CloseToDoor), lJ.e(lnCorridor)})[fi, ^ 2 ] 

A AG({Hi(lnCorridor)})[fo,ii] 

A AG({Lo(TimeOut)})[to, f] 

Assuming reasonable settings of the Hi and Lo thresholds h, I, the following substitutions 
of the time variables to time-points yield the mapping into the activation histories in 
Fig-13 t = 28 (right outside the figure), to = 3, = 12, t2 = 16, t^ = 20, t^ = 24. As 

a result, we extract In Room [24]. 

This substitution is not unique. For example, postponing to until 5 or having ti earlier 
at 9 would also work. This point leads to the process of chronicle recognition: given 
a working BBS, permanently producing activation values, how are the given defining 
chronicles of facts checked against that activation value data stream to determine whether 
some fact starts to hold? 

The obvious basis for doing this is to keep track of the qualitative activations as 
they emerge. That means, for every behavior, there is a process logging permanently the 
qualitative activations. For those of type HI and Lo, the sufficiently long time periods of 
the respective behavior activation above and below the h, I thresholds, resp., have to be 
recorded and, if adjacent to the current time point, appropriately extended. This would 
lead automatically to identifying qualitative activations of types Hi and Lo with their 
earliest start point, such as = 3 for Hl(lnCorridor) in the example above. Qualitative 
activations of types 'ffe and Ije are logged iff their definitions (eqs.[E|and[0 resp.) are 
fulfilled in the recent history of activation values. As this logging process is local to 
every behavior, the complexity is linear 0{B) in the number B of behaviors. 

Qualitative activation logs are then permanently analyzed whether any of the existing 
defining chronicles are fulhlled, which may run in parallel lo Ihe ongoing process of 



Extracting Situation Facts from Activation Value Flistories in Behavior-Based Robots 



315 



logging the qualitative activations. An online version of this analysis inspired hy ltGha96l 
would attempt to match the flow of qualitative activations with all defining chronicles 
c hy means of matching fronts that jump along c’s internal interval boundary points ti 
and try to bind the next time point as current matching front such that the recent 
qualitative activations fit all sub-intervals of c that end in fi+i. Note that more than one 
matching front may be active in every defining chronicle at any time. A matching front 
in c vanishes if it reaches the end point t (the defining chronicle is true), or else while 
stuck at ti is caught up by another matching front at ti, or else an activation gestalt over 
an interval ending at is no longer valid in the current qualitative activation history. 

For complexity considerations, assume that C defining chronicles are defined that 
involve a maximum of — 1 internal interval boundary points. Assume further that A 
is the maximal number of qualitative activations occurring in single activation gestalt 
conjuncts of all defining chronicles. (A is bounded by the number B of behaviors.) Then, 
one cycle of the online matching of defining chronicles runs in 0{ACN) time. 

Practically, the necessary computation may be focused by specifying for each defin- 
ing chronicle a trigger condition, i.e., one of the qualitative activations in the definition 
that is used to start a monitoring process of the validity of all activation gestalts. For 
example, in the InRoom definition above, 'fl'e(GoThruDoor), as occurring in the [fa, t 4 ] 
interval, might be used. Note that the trigger condition need not be part of the earli- 
est activation gestalts in the definition. On appearance of some trigger condition in the 
qualitative activation log, we try to match the activation gestalts prior to the trigger with 
qualitative activations in the log file, and, if successful, verify the gestalts after the trigger 
condition in the qualitative activations as they are being logged. 




0 2 4 6 6 10 12 14 16 1» 20 22 24 




Fig. 4. Robot example 2: final state and activation curves. See text for explanations. 
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To give an example where the derivation of the InRoom fact fails, consider Fig.0 
The scenario is like before, i.e., the robot starts at the lower right corner, driving upward 
and trying to enter any door large enough. Different to Fig.[IJ no obstacle is present at 
the beginning, and while the robot tries to enter into the detected door, another robot 
comes from within the room and blocks it. Fig. fright shows the scene after the robot 
has failed to enter the door, and left are the respective activation values. 

Like before, while InCorridor (fg = 3), the CloseToDoor and TurnToDoor acti- 
vations rise with InCorridor falling {t\ = 9,^2 = 13); then, TurnToDoor is Hi, while 
InCorridor is Lo (fg = 20). But then, mischief strikes. After the long high period 
of TurnToDoor, TimeOut jumps up, terminating its Lo period, before GoThruDoor 
has risen. In consequence, cannot be bound, and the fact In Room not extracted. If 
'fl'e(GoThruDoor) was used as a trigger condition in the first place, then no unnecessary 
matching effort was wasted. 

Some more general remarks are in place here. Defining chronicles exclusively in 
terms of activation gestalts is a special case that we have used in this paper to keep 
matters focused. In general, the obvious other elements may be used for defining them: 
sensor readings at some time points (be they physical sensors or sensor filters), and 
the validity of symbolic facts at a time point or over some time interval. Our intention 
is to provide fact extraction from activation values as a main source of information, 
not the exclusive one. That type of information can be added to the logical format of 
a chronicle definition as in O. For example, if the exact time point of entering a 
room with the robot’s front is desired as the starting point of the In Room fact, then 
this might be determined by the time within the interval [^ 4 , t] (i.e., within the decrease 
of the GoThruDoor activation) where some sensor senses open space to the left and 
right again. As another example, assume that the fact Al{DoorA) for the door to some 
room A may be in the fact base (as derived from a normal localization process). Then 
At(Door^)[f 4 ] could be added to the defining chronicle fT^ above to derive not only 
lnRoom[t], but more specifically lnRoom(A)[f]. 

The fact extraction technique does not presume or guarantee anything about the con- 
sistency of the facts that get derived over time. Achieving and maintaining consistency, 
and determining the ramifications of newly emerged facts remain issues that go beyond 
fact extraction. Pragmatically, we would not recommend to blindly add a fact as true 
to the fact base as soon as its defining chronicle has been observed. A consequent of 
a recognized defining chronicle should be interpreted as evidence for the fact or as a 
fact hypothesis, which should be added to the robot’s knowledge base only by a more 
comprehensive knowledge base update process, which may even reject the hypothesis 
in case of conflicting information. A possible solutions would be to add some integrity 
constraints to the defining chronicles. However, this is not within the scope of this paper. 



6 Discussion 



A physical agent’s perception categories must to some degree be in harmony with its 
actuator capabilities — at least in purposively designed technical artifacts such as working 
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autonomous robotsflOur approach of extracting symbolic facts from behavior activation 
merely exploits this harmony for intertwining control on a symbolic and a reactive level 
of a hybrid robot control architecture. 

The technical basis for the exploitation are time series of behavior activation values. 
We have taken them from a special type of behavior-based robot control systems (BBSs), 
namely, those consisting of behaviors expressed by nonlinear dynamical functions of a 
particular form, as described in Sec. El The point of having activation values in BBSs 
is not new; it is also the case, e.g., for the behavior-based fuzzy control part underlying 
Saphira IKMSR97II . where the activation values are used for context-dependent blending 
of behavior outputs, which is similar to their use in our BBS framework. Activation values 
also provide the degree of applicability of the corresponding motor schemas in IArk981 
p. 141]. 

The activation values of a dynamical system-type BBS are well-suited for fact ex- 
traction in that their formal background in dynamical systems theory provides both the 
motivation and the mathematical inventory to make them change smoothly over time — 
compare, e.g., the curves in FiguresE|and0]with the ragged ones in |SRK99[ Fig. 5.10]. 
This typical smoothness is handy for defining qualitative activations, which aggregate 
particular patterns in terms of edges and levels of the curves of individual behaviors, 
which are recorded as they emerge over time. These then serve as a stable basis for chron- 
icle recognition over qualitative activations of several behaviors. Note, however, that this 
smoothness is a practical rather than a theoretical issue, and other BBS approaches may 
serve as bases for fact extraction from activation values. 

We want to emphasize that the activation values serve two purposes in our case: 
first, their normal one to provide a reliable BBS, and second, to deliver the basis for 
extracting persistent facts, based on their distinctive patterns. With the second use, we 
save the domain modeler a significant part of the burden of designing a complicated 
sensor interpretation scheme only for deriving facts. The behavior activation curves, as a 
by-product coming for free of the behavior-based robot control, focus on the environment 
dynamics, be it induced by the robot itself or externally. By construction, these curves 
aggregate the available sensor data in a way that is particularly relevant for robot action. 
We have argued that this information can be used as a main source of information about 
the environment; other information, such as coming from raw sensor data, from dedicated 
sensor interpretation processes, or from available symbolic knowledge, could be used 
in addition. 

As activation values are present in a BBS anyway, it is possible to "plug-in" the fact 
extraction machine for a deliberative component to an already existing behavior system 
like the DD control system in llBGG~*~99ll . Yet, if a new robot control system is about to 
be written for a new application area, things could be done better, within the degrees of 
freedom for variations in behavior and domain model design. The ideal case is that the 
behavior inventory and the fact set is in harmony in the sense that such facts get used 
in the domain model whose momentary validity engraves itself in the activation value 
history, and such behaviors get used that produce activation values producing evidence 
for facts. For example, a single Wall Follow behavior working for walls on the right and 

^ We do not speculate about biological agents in this paper, although we would conjecture that 
natural selection and parsimony strongly favor this principle. 
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on the left, may be satisfactory from the viewpoint of behavior design for a given robot 
application; for fact extraction, it may be more opportune to split it into FollowLeftWall 
and FollowRightWall, which would be equally feasible for the behavior control, but 
allows more targeted facts to be deduced directly. 

Apart from such design-level interdependencies, which are non-trivial, but not special 
for our approach, we are aiming at a control architecture with a deliberative and a 
behavior-based part as two abreast modules with no hierarchy, as sketched in |HJZM98| . 
The fact extraction scheme leaves the possibility to un-plug the deliberative part from 
the robot control, which we think is essential for robustness of the whole robot system. 

Our technique is complementary to anchoring symbols to sensor data as described in 
omn . It differs from that line of work in two main respects. First, we use sensor data 
as aggregated in activation value histories only, not raw sensor data. Second, we aim 
at extracting ground facts rather than establishing a correspondence between percepts 
and references to physical objects. The limit of our approach is that it is inherently 
robot-centered in the sense that we can only arrive at information that has to do directly 
with the robot action. The advantage is that, due to its specificity, it is conceptually and 
algorithmically simpler than symbol anchoring in general. 



7 Conclusion 



We have presented a new approach for extracting information about symbolic facts from 
activation curves in behavior-based robot control systems. Updating the symbolic en- 
vironment situation is a crucial issue in hybrid robot control architectures in order to 
bring to bear the reasoning capabilities of the deliberative control part on the physical 
robot action as exerted by the reactive part. Unlike standard approaches to sensing the 
environment in robotics, we are using the information hidden in the temporal develop- 
ment of the data, rather than their momentary values. Therefore, our method promises 
to yield environment information that is complementary to normal sensor interpretation 
techniques, which can and should be used in addition. 

We have presented the technique in principle as well as in terms of selected demo 
examples in a robot simulator, which has allowed to judge the approach feasible and 
to design the respective algorithms. The computational complexity of the recognition 
process is in 0{BCN), where B is the number of behaviors, C the number of chronicle 
definitions, and N the maximal “length" of a chronicle definition in terms of intermediate 
time points internal to the chronicle definition. 

The approach will be applied in the context of the hybrid robot control architecture 
DD&P 1HJZM98I to generate an important part of the information that is used to update 
the symbolic world model from the sensor data stream. Work is ongoing towards a 
physically concurrent implementation of DD&P on physical robots, as described in 
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Abstract. One of the major problems in clausal theorem proving is the 
control of the proof search. In the presence of equality, this problem is 
particularly hard, since nearly all state-of-the-art systems perform the 
proof search by saturating a mostly unstructured set of clauses. We de- 
scribe an approach that enables a superposition-based prover to pick 
good clauses for generating inferences based on experiences from pre- 
vious successful proof searches for other problems. Information about 
good and bad search decisions (useful and superfluous clauses) is auto- 
matically collected from search protocols and represented in the form of 
annotated clause patterns. At run time, new clauses are compared with 
stored patterns and evaluated according to the associated information 
found. We describe our implementation of the system. Experimental re- 
sults demonstrate that a learned heuristic significantly outperforms the 
conventional base strategy, especially in domains where enough training 
examples are available. 



1 Introduction 

The last few years have seen an impressive improvement in the power of first- 
order theorem provers, and a corresponding increase in the use these systems 
in research and development. They are being used for the verification of pro- 
tocols i»chfi7iWeififii(TPnni . and the retrieval of mathematical theorems 
or software components ir?n from libraries. Theorem provers are also used 
to synthesize larger programs from standard building blocks and to prove the 
correctness of the resulting program systems fSWL+94lfim.MDHj . For these pur- 
poses, various systems have been embedded into large r, interactive proof systems 
like ILF |n(IHWfi7| . KIV or VSE fHLS+Qfij . 

The most visible success of automatic theorem provers (or ATP systems) 
today is the celebrated proof of the Robbins algebra problem by EQP |McC97j . 
Successes like this demonstrate the power of current theorem proving technology. 
However, despite the fact that ATP systems are able to perform basic operations 
at an enormous rate, and can solve most simple problems much faster than any 
human expert, they still fail on many tasks routinely solved by mathematicians, 
even if these tasks can be encoded in first-order logic in a natural way. Moreover, 
many of the more impressive successes require an experienced human user who 
selects a suitable prover configuration, often by trial and error. 
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The main reason for this is that a theorem prover has to search for a proof 
in a usually infinite search space with a very high branching factor, i.e. a very 
high number of possible choices at each choice point. Much previous work in 
theorem proving has been targeted at the development of refined calculi that 
reduce this branching factor by restricting the number of possible inferences. 
However, the semi-decidability of the underlying problem for most interesting 
logics restricts the potential for this approach, and even the most refined calculi 
typically are highly non-deterministic. This is particularly true for calculi that 
deal with equality, as the congruence properties of this relation imply a very 
large number of possible inferences. Moreover, all calculi for equational logic 
that have been successfully implemented and show good performance operate on 
unstructured clause sets, and lack the inherent goal-orientedness of e.g. model 
elimination calculi or the set-of-support strategy for non-equational logic. 

Most current theorem provers therefore use a small set of highly parameter- 
ized heuristic evaluation functions to guide the proof search. The selection of a 
proper evaluation function and set of parameters for a given problem (or problem 
domain) is based on experience of the human user, often supported by large and 
tedious sets of experiments. It is often encoded in a so-called automatic mode, 
where the prover selects one of a relatively small set of pre-configured strate- 
gies based on properties of the current proof problem. While such automatic 
configuration can improve the performance of a prover if used by a non-expert, 
the development of new strategies is still a manual process that requires a lot 
of work and attention by an expert user or even developer, and is hence very 
expensive in terms of man power. 

In this paper we suggest a way to automatically learn information about good 
and bad search decisions from examples of successful proof searches, and to use 
the learned knowledge to define new search control heuristics. A proof search is 
represented by a clause dependency graph, and clauses contributing to a proof 
as well as superfluous clauses are identified. A suitable subset of these clauses 
is used to represent good and bad search decisions. We encode these clauses as 
patterns that abstract from irrelevant information and use the information about 
the value of a decision in later proof searches to compute an informed evaluation 
of new clauses. 

For a broad overview of related works, see [I . Our own previous work 

in this direction (see [II )Shfi)l )S0()] ) was restricted to the unit-equational case, 
where the suitable encoding of search decisions is much easier. Moreover, these 
previous approaches were based on positive search decisions only, and, due to the 
relatively small number of unit-equational proof problems publically available, 
we did not perform an experimental evaluation of the same rigidity. Other related 
work has been reported by Fuchs |Fuc96IFuc'^ . The main differences are that 
Fuchs again focuses on the unit-equational case, that his clause generalizations 
are weaker, and that he tries to replay search decisions from a single proof 
problem. Our approach, on the other hand, gathers experience from an arbitrary 
number of proof searches. 
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We start the remainder of this paper with a very short introduction to 
superposition-based theorem proving and the organization of the search process. 
In the next section we describe our approach to learning and cover knowledge 
acquisition, representation, and application. Section 2| contains experimental re- 
sults, and we conclude in section 0 



2 Superposition-Based Theorem Proving 



The superposition calculus |RG94IB(;98| and its variants are generally recog- 
nized as the most powerful calculi currently available to tackle theorem proving 
problems with equality. The well-known prover SPASS ^AR+99| and our own 
system E jSch99ISchfl1| are based on particular instances of this calculus, and 
all other leading saturating theorem provers incorporate at least some important 
features of it. We will give a very short introduction, concentrating on aspects 
important for search control. 



2.1 Calculus 

The superposition calculus is a refutational calculus. It works on a set of clauses, 
and tries to show the unsatisfiability of the clause set by deriving the empty 
clause. A clause in this calculus is a multi-set of literals (positive or negative 
equation^ and is interpreted as the universally quantified disjunction of these 
equations. An equation is an unordered pair of first-order terms. Generating 
inferences add new clauses to the set, contracting inferences modify or remove 
existing clause. A theorem proving derivation is a sequence of generating and 
contracting inferences applied to a set of clauses. Most inference types are re- 
stricted by a simplification ordering on terms, which is extended to equations 
and clauses. 

The most important generating inference in any superposition calculus is 
the superposition inference, a paramodulation inference restricted to maximal 
terms in maximal literal^. It can be seen as lazy conditional rewriting applied 
to instances of clauses resulting from unification. 

Despite the restrictions imposed by ordering and literal selection, superpo- 
sition inferences are typically responsible for more than 99% of all clauses gen- 
erated during a proof search. In addition to superposition inferences, special 
factoring inferences and equality resolution inferences are necessary for the com- 
pleteness of the calculus. However, the number of clauses generated by these 
inferences typically is miniscule. 

One of the major strengths of the superposition calculus is its compatibility 
with a wide range of contracting inferences. Probably the most important of 
these contracting inferences is the rewriting of clauses i.e. the use of orientable 

^ Non-equational atoms are encoded as equations. 

^ In many cases, inferences can be further restricted by literal selection, i.e the restric- 
tion of paramodulation inferences to a single negative literal. 
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instances of positive unit clauses to replace terms with smaller terms. Other con- 
tracting inferences are the removal of trivial or duplicated literals. Tautological 
or subsumed clauses can be eagerly removed without affecting the completeness 
of a proof search (assuming a suitable definition of subsumption). These con- 
tracting processes typically take up a large part of the total work during a proof 
search, and are totally indispensable for the efficient work of the prover. 

2.2 Organizing the Proof Search 

The superposition calculus is complete, i.e. for any unsatisfiable clause set there 
is a theorem proving derivation that derives the empty clause. However, in order 
to guarantee that the empty clause is found, certain fairness constraints are 
imposed upon the search. A sufficient way to ensure that these constrains are 
met is to eventually perform all generating inferences between persistent clauses, 
i.e. to ensure that no possible inference is delayed forever. This is achieved by 
different variants of the given-clause algorithm. This algorithm uses two sets of 
clauses, a set of unprocessed clauses, which initially contains the input clauses, 
and a set of processed clauses, which initially is empty. The algorithm repeatedly 
picks one of the unprocessed clauses, performs all generating inferences between 
this given clause and all clauses in the processed set, adds the newly produced 
clauses into the set of unprocessed clauses, and moves the given clause to the 
set of processed clauses. 

The selection of the given clause is typically done by a heuristic evaluation 
function which rates each clause. The most widely used and successful heuristics 
are based on symbol counting, and assign a low weight to clauses with a low 
number of symbols. This heuristic may be interleaved with a first-in first- out 
strategy that always picks the oldest remaining clause. 

Different implementations of the given-clause algorithm differ mainly in the 
way in which contracting inferences are performed. In our version, the given 
clause is rewritten with the processed clause set and checked for subsumption 
after being picked. Then we purge clauses that can be subsumed by or rewritten 
with the given clause from the set of processed clauses. In addition, newly gen- 
erated clauses are rewritten and checked for obvious tautologies. Other variants 
of the algorithm also keep the list of unprocessed clauses in normal form or use 
generated unit clauses more eagerly for rewriting processed clauses (backward 
contraction). We will abstract from contracting inferences in the following, for a 
discussion of their effect and handling see IRMI . 

3 Learning Search Control Knowledge 

We have performed a detailed analysis of the given clause algorithm, supported 
by experimental data, and found that the selection of the given clause is the 
most critical choice point in this algorithm EEm. In a typical proof search 
today, a high-performance prover generates up to a few million clauses within 
5 minutes. Of these, up to about 10000 are processed. However, typically much 
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less than 200 clauses are needed for the proof. If we assume an optimal oracle, 
processing these 200 clauses only takes minimal time for modern theorem provers 
- typically less then a second, and in virtually all cases less than 5 seconds. 

Based on these results, we consider the choice of the next clause to process 
the most crucial choice point in the given-clause algorithm. We try to improve 
the proof search by learning good evaluations for clauses based on experiences 
from previous proof searches. 

In order to be able to meet this goal, we have developed a general framework 
for incorporating feedback from previous proof searches into the selection of the 
given clause. The corresponding learning theorem prover is organized as follows: 

— In a first phase, a protocol of the inference steps is turned into a proof 
derivation graph. This graph is analyzed, and clauses contributing to the 
proof as well as clauses that are in certain sense close to the proof are 
selected as training examples. 

— These clauses are then transformed into annotated patterns, where a pattern 
is a unique representation for a class of structurally identical clauses, and 
where an annotation describes the role of the clauses corresponding to the 
pattern in previous proof searches. The resulting sets of patterns are stored 
in a knowledge base and indexed by a feature vector describing the original 
proof problem. 

— In the application phase, we use this feature vector to decide on a set of 
proof experiences to use to guide the new proof search. The corresponding 
patterns are recalled and fed into a learning algorithm. Newly generated 
clauses are evaluated using the resulting learned knowledge representation. 

In the instance of the general framework described in this paper, the learn- 
ing algorithm simply stores the clause patterns with their corresponding anno- 
tations. If a new clause matches a known pattern, its standard evaluation is 
modified based on the annotations. 

Our implementation of the learning process is fully automatic. The only 
human intervention necessary is the a-priory selection of parameters for both 
the theorem prover and the analysis module that determines which clauses are 
used to represent a given proof search. 

Fig.in shows a general sketch of this learning cycle as implemented in our 
prover E. We will discuss the different aspects of this cycle in the next sections. 

3.1 Knowledge Acquisition 

The first problem is to identify which search decisions from a given proof search 
should be used as input for a learning prover. As we wrote above, the starting 
point for our learning system is a complete listing of all inferences performed by 
the prover during a successful proof search. Such a listing basically consists of a 
number of steps, each of which contains a clause (the result of the step) and a 
justification for this clause. A justification is either “initial” for clauses from the 
problem specification, or a description of the inference generating this clause, 
including pointers to all clauses used in the premise of the inference. 
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Problem specification 




Fig. 1. Architecture of the learning theorem prover 



An inference listing defines a directed clause dependency graph. In this graph, 
nodes correspond to the clauses (where different occurrences of the same clause 
are considered to be distinct objects and are represented by different nodes) and 
edges go from the premises of an inference to the conclusion. A path from an 
initial clause to the empty clause is called a proof path, clauses on a proof path 
are called contributing (to the proof) and all other clauses are called superfluous. 
The subgraph containing exactK the nodes and edges of proof paths is called 
the proof object or simply prooj^. 

In a typical non-trivial proof search, superfluous clauses outnumber con- 
tributing clauses by a factor of at least 100. To keep the knowledge base man- 
ageable, and to balance the number of contributing and superfluous clauses we 
use as training examples, we select an appropriate number of superfluous clauses 
from those that are close to the proof, i.e. that can be derived in at most a small 
number of inferences from the contributing clauses. Additionally, we discard 
clauses that have never been selected as the given clause, as these clauses did 
not have a chance to contribute to a proof at all, and hence we have no infor- 
mation on the impact they would have had on the proof search. 

^ While proofs are often considered to be trees, generating provers reuse intermediate 
results, and thus each node (corresponding to a clause) can be referenced more than 
once, resulting in a general directed acyclic graph structure. The customary proof 
tree can be generated by recursively duplicating subgraphs referenced more than 
once, thus unfolding the graph. However, as this typically leads to an exponential 
explosion of the number of nodes, we consider the graph representation to be not 
only more natural, but also more suitable for our purposes. 



326 



S. Schulz 



All selected clauses are annotated with information about their role in the 
proof process. We found that the clause status (contributing or superfluous) and 
the distance of the clause from the closest proof path are useful for assigning 
evaluations to clauses. 

3.2 Knowledge Representation 

The next problem we have to tackle is to find an adequate representation of 
search decisions. This representation has to strike a fine line between generality 
and specificness. Equivalent search decisions should be represented by similar 
structures, but significantly different ones should still be distinct. 

We achieve this compromise by the generalization of clauses into clause pat- 
terns. This serves two purposes. First, remember that clauses are multi-sets of 
literals, and are hence unordered. Thus, there are 2" different ways to order 
the literals in a clause with n literals. Moreover, equational literals are again 
unordered pairs of terms, and can be arranged independently in each literal. 
Thus, any given clause with n literals has n\ 2” different syntactic representa- 
tions. We want to have a unique representation for all these strictly equivalent 
but syntactically different clause representations. 

Secondly, the transformation into patterns abstracts from the given signa- 
ture of the problem. In our experience, specifications of problems from the same 
domain are often encoded using different function symbols. Moreover, there are 
analogous substructures even within a single specification. As an example, con- 
sider algebraic fields, which contain both an additive and a multiplicative group. 
We are therefore convinced that the loss of information resulting from the ab- 
straction of signature information is more than compensated for by the ability to 
generalize to analogous situations. Earlier experiments for the unit-equational 
case as well as the experimental results presented in this paper 

support this conviction. 

We achieve these two goals by systematically replacing function symbols 
with new symbols from the set {fij\i,j S N} (where the first index encodes 
arity, the second one distinguishes between different symbols of the same arity), 
and by reordering the terms and literals in the clause representation, until a 
representation is found that is minimal in a suitable ordering. 

We use a modified lexicographic ordering that first compares terms based only 
on their size and structure (ignoring the symbols) and then compares terms lex- 
icographically. The ordering is then lexicographically lifted to literal and clause 
representations. The search for the minimal clause representation is conducted 
using a conventional backtracking algorithm, which in the practical implemen- 
tation is supported by a pre-ordering of terms and literals with the part of the 
ordering that is independent of function symbols and depends on the structure 
of terms only. The final result is then encoded as a first order term over an 
extended signature. 

We use the reserved symbols eqj 2 and neqj 2 to encode positive and negative 
equations, respectively. The disjunction of the literals of a clause is encoded by 
using the reserved symbols or j 2 and nUj 0 as standard list constructors. 
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As an example, the clause 



b = f{a, a)\/ a ^ b 



is represented by the pattern 

or{eq{f 2 o{foo, /oo), /oi), or{neq{foo, /oi), nil)) 

The knowledge base simply stores patterns of clauses with the annotations 
computed in the proof search analysis. Patterns are indexed by a vector of 15 
feature values describing aspects of the original problem specification, and by 
arity frequency vectors for function symbols and predicate symbols. Features 
used include e.g. the number of unit, Horn and general clauses, the average size 
and depth of terms, and the average number of positive and negative literals 
per clause. The arity frequency vectors describe how many symbols of each arity 
are present in the problem signature. For a detailed description of the problem 
selection mechanism see jSchOO) . 

All relevant data is stored in the form of plain ASCII text files in a subdi- 
rectory on the hard disk. While it is possible to modify the knowledge base by 
hand, we have implemented tools to create a knowledge base and to automati- 
cally insert new or remove old proof examples. 

3.3 Knowledge Application 

The application phase is concerned with two main problems: The identification 
of suitable proof experiences and the application of the corresponding knowledge 
to the new proof search. 

If a new proof problem is given to the learning theorem prover, the corre- 
sponding feature vector is computed. The prover then uses a standard distance 
measure (modified Euclidean distance) to determine a set of similar problem^ 
All clause patterns from selected proof experiences are stored into a pattern 
cache organized as a splay tree ISISSl, a memory-efficient data structure suit- 
able for very fast data retrieval. If the same pattern occurs more than once in the 
knowledge base, the corresponding annotations are combined. As a result, each 
of the patterns in the tree carries a tuple (p, d ) , where p describes the number 
of proofs the clauses corresponding to the pattern participated in, and d is the 
average distance of these clauses from the proof. 

New clauses to be evaluated are transformed into patterns as well. If the 
resulting pattern is found in the cache, we use the annotation of the pattern 
to compute the evaluation. Otherwise, we use an annotation {0,dmax)^ where 
{dmax) is the largest distance in the cache, increased by one. In order to increase 
the stability of the prover and to help the system to fill gaps in the acquired 
knowledge, we combine a conventional heuristic with the learned knowledge. 

For pattern memorization, we get no benefit from selecting only similar problems, 
and hence always use all available training examples (see section 0 for a short discus- 
sion). However, if we use more sophisticated versions of term space mapping EcEOIl, 
this selection becomes important. 
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Consider a clause C to be evaluated with an annotation of (p, d) . A clause 
with a large value of p should get a better than standard evaluation (it has 
contributed to many proofs). A clause with a large value of d, on the other 
hand, usually has not contributed to proofs, and should receive a worse eval- 
uation. Now assume that w is the weight assigned by a standard symbol 
counting evaluation function. The weight assigned by the learning heuristics 
is lw{C) = w(l -I- winorm (wpP + Wdd)), where p and d are normalized by divid- 
ing p and d by the largest corresponding values in any annotation in the pattern 
cache, where Wp and Wd determine the relative influence of these two variables, 
where norm rescales all evaluations for annotations in the cache to the interval 
[0; 1], and where finally wi determines the overall influence of the learning com- 
ponent. The double renormalization effect allows us to independently control the 
influence of the two parameters p and d, as well as the relative weight given to 
the conventional and the learned parts of the evaluation. 

Please note that for positive values of wi the resulting strategy is fair if the 
conventional base strategy is, i.e. it will never prefer an infinite number of clauses 
to any given clause. 

4 Experimental Results 

We have implemented the above concepts as a special case of our learning the- 
orem prover E/TSM. To evaluate the approach, we have tested the resulting 
system on the set of all 3558 clause normal form (CNF) problems from the 
TPTP problem library jbb Y94lbb97j . version 2.3. (0. 

Clause normal form problems in the TPTP are either unsatisfiable, which 
implies that the underlying first-order problem does have a proof, or satisfiable, 
which means that a model exists that can serve as a counterexample for the 
original first-order problem. A problem can be proved by a prover if it can show 
the unsatisfiability. Alternatively, a prover can (in our case rarely) show that 
no proof is possible, i.e. that a model for the specification exists. We consider 
both of these case to be a success of the proof search. Termination of the proof 
search without a definitive answer about the state of the problem, either due to 
reaching a time limit or due to running out of some other limited resource, are 
considered failures. 

To test the success of learning, we compare the learning heuristic to a stan- 
dard symbol counting heuristic (which always picks the clause with the least 
number of symbols). Both heuristics are interleaved with first-in-first-out (al- 
ways pick the oldest clause) using a pick-given ratio of 5 to 1. For the sym- 
bol counting heuristic, this combination corresponds to a widely used standard 
heuristic implemented e.g. in Otter |MW97| and Vampire fHV H9| . E controlled 

® The results presented in the following have been obtained in compliance with the 
guidelines for use of the TPTP. TPTP input files were unchanged except for removal 
of equality axioms and syntax transformation. All experiments were performed on a 
cluster of SUN Ultra 60 workstations running at 300 MHz. Memory was limited to 
192 megabyte for each proof attempt. 
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by this conventional heuristic can prove 1602 out of the 3558 CNF problems in 
TPTP within a 300 second time limit on our hardware. It also finds 88 models, 
for total of 1690 successes. 

The learning evaluation function uses the same symbol counting heuristic as 
a base and uses values of 5 for wi, -20 for Wp and 20 for Wd (note that only 
the relative size of Wp and Wd are relevant due to the normalization). After 
preliminary experiments, we have always chosen all problems in the knowledge 
base as training examples. It turns out that any strong decrease in the number 
of problems used also impairs the performance, while the presence of more, 
probably very different problems does not hurt the pattern memorization based 
heuristic at all. This may be due to the fact that very different proof problems 
also generate syntactically different clause patterns, and hence do not influence 
the proof search at all. 

In all cases, we have used a Knuth-Bendix ordering with constant weight for 
all symbols and a precedence that is induced by the symbol arities (symbols with 
a higher arity are bigger than symbols with a lower arity, order between symbols 
of the same arity is chosen at random) . We also used a literal selection strategy 
that always selects the negative literal with the largest size difference between 
the two terms in non-positive clauses. 

To demonstrate the ability of the learning heuristic to generalize to unseen 
problems, we have adapted 10 fold stratified cross validation, a standard tech- 
nique from the field of machine learning. The set of all proof problems is par- 
titioned into 10 approximately equal sized sets (or folds). Within each set, the 
number of problems that can be solved by the conventional heuristic is again 
approximately equal. We then use the proofs from nine out of the ten folds as 
training examples for the learning heuristic and apply it to the remaining fold. 
In particular, this means that none of the new proof problems is already known 
to the prover. The resulting knowledge bases each contain about 1450 training 
examples, with about 70000 clauses. These are mapped onto about 14000 dis- 
tinct clause patterns. The knowledge bases have a size of about 5 megabyte for 
the functional part and of approximately 20 megabyte if we include the full, 
uncompressed set of training data for archival purposes. 

Table E shows the results of the cross evaluation. We again used a time limit 
of 300 seconds for each proof attempt. As we can see, in each fold the learning 
heuristic performs better than the conventional heuristic. If we consider the set 
of all problems, this improvement is statistically significant. We can also see that 
all of the improvement is due to the finding of more proofs. Indeed, the learning 
heuristic does loose one model compared to the base heuristic. 

If we consider the performance of the 10 learning heuristics as a whole, we find 
that there are 19 problems that are only proved by the conventional heuristic, 
but not by the learning heuristic used on the given problem. On the other hand, 
there are 138 problems that are solved by the appropriate learning heuristic, but 
not by the conventional one. Thus, while the learning heuristics do not prove a 
strict superset of the problems solved by the conventional one, they can solve a 
fairly large number of new, hard problems. 
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Table 1. Results of 10-fold stratified cross validation 



Fold 


C 

Proofs 


Convention 

Models 


al 

Successes 


Proofs 


Learning 

Models 


Successes 


1 


161 


12 


173 


180 


12 


192 


2 


161 


7 


168 


172 


7 


179 


3 


160 


12 


172 


167 


12 


179 


4 


160 


10 


170 


171 


10 


181 


5 


160 


6 


166 


173 


6 


179 


6 


160 


14 


174 


168 


13 


181 


7 


160 


6 


166 


175 


6 


181 


8 


160 


5 


165 


165 


5 


170 


9 


160 


10 


170 


179 


10 


189 


10 


160 


6 


166 


171 


6 


177 


Average 


160.2i0.422 


8.8±3.190 


169.0i3.266 


172.li4.886 


8.7i3.020 


180.8i6.088 


Total 


1602 


88 


1690 


1721 


87 


1808 



We can also compare the different heuristics on subsets of the TPTP: Prob- 
lems containing unit clauses only, non-unit Horn problems, and problems with 
general clauses. Each class is once more split according to the presence of an 
equational sub-theory. Table El shows the results. 

Again we can see that the learning heuristic is superior nearly across the 
board. However, the gain is most significant for the case of non-horn-problems 
with equality (which also is generally considered to be the hardest class of prob- 
lems in the TPTP), and is fairly small for unit problems. This may be corre- 
lated with the size of the corresponding class. The improvements achieved by the 
learning heuristic increase with the size of the problem class, which supports the 
plausible idea that more suitable training examples lead to better performance 
on unknown problems. 



Table 2. Number of successes by problem class 



Problem class 


Size 


Conventional 


Learning 


Successes gained 


Unit, no equality 


11 


11 


11 


0.00% 


Unit with equality 


447 


289 


292 


1.04% 


Horn, no equality 


609 


464 


492 


6.03% 


Horn with equality 


507 


291 


308 


5.84% 


General, no equality 


766 


307 


333 


8.47% 


General with equality 


1218 


328 


372 


13.41% 


Total 


3558 


1690 


1808 


6.98% 



Finally, we can compare both heuristics on the TPTP domains. A TPTP 
domain is a set of problems dealing with similar structures. However, TPTP 
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domains are usually more diverse than typical application domains, because 
TPTP domains have been created a posteriori to organize a large number of 
existing proof problems, and do not result from a systematic axiomatization of 
a given application area. 



Table 3. Number of successes by TPTP domain 



Problem class 


Size 


Conventional 


Learning 


Successes gained 


ALG 


10 


2 


2 


0.00% 


ANA 


19 


2 


2 


0.00% 


BOO 


68 


55 


60 


9.09% 


CAT 


58 


54 


53 


-1.85% 


CID 


4 


1 


1 


0.00% 


CIV 


14 


11 


11 


0.00% 


COL 


160 


94 


94 


0.00% 


COM 


6 


5 


5 


0.00% 


FLD 


281 


24 


53 


120.83% 


GEO 


165 


98 


101 


3.06% 


GRA 


1 


1 


1 


0.00% 


GRP 


376 


254 


251 


-1.18% 


HEN 


64 


62 


63 


1.61% 


KRS 


17 


17 


17 


0.00% 


LAT 


35 


12 


16 


33.33% 


LGL 


503 


262 


296 


12.98% 


LDA 


23 


19 


19 


0.00% 


MGT 


0 


0 


0 


- 


MSC 


13 


10 


9 


-10.00% 


NUM 


309 


33 


37 


12.12% 


PLA 


30 


5 


5 


0.00% 


PRV 


9 


8 


8 


0.00% 


PUZ 


60 


53 


53 


0.00% 


RNG 


98 


41 


45 


9.76% 


ROB 


36 


14 


14 


0.00% 


SET 


695 


171 


212 


23.98% 


SYN 


480 


377 


376 


-0.27% 


TOP 


24 


5 


4 


-20.00% 


Total 


3558 


1690 


1808 


6.98% 



Table El shows the results by domain. Now we see a more diverse picture. 
In many domains, especially smaller ones, learning does not improve the perfor- 
mance of the prover much, or even leads to slightly worse performance (although 
this is typically not significant). Most of the gain comes from a couple of larger 
domains. FLD, LAT and SET are particularly impressive. Again, we get the 
impression that an increased number of suitable training examples improves the 
performance of the learning heuristic. This makes it likely that the learning 
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heuristics are particularly useful in domains like verification or software reuse, 
where a large number of similar problems have to be solved over and over again, 
possibly with slight modifications. 

The successes of the learning heuristics are achieved despite a significant 
computational overhead. It takes about 15 seconds to read the knowledge base 
into memory, determine the set of similar problems, and to prepare the pattern 
cache with pre-computed evaluations. Moreover, the inference rate is about 20% 
lower as for the less complex conventional strategies. Profiling shows that most of 
this overhead is spent on the transformation of new clauses into patterns. As this 
operation depends only on the single clause, and not on the total size of the clause 
database (as e.g. superposition and subsumption), this cost is relatively less 
significant for large problems, and will likely become less important as advances 
in hardware allow us to deal with even bigger search spaces. 



5 Conclusion 

In this paper, we have described a fully automatic learning theorem prover based 
on the memorization of clause patterns from previous proof searches. The learn- 
ing heuristics were able to significantly outperform a conventional base heuristic, 
especially on hard problems. Our results show that clause pattern memorization 
is an adequate method for learning search control knowledge in domains where 
enough previous proof experiences are available. We find that time spent on good 
search guidance is usually well spent. While our results have been obtained with 
a superposition-based prover, there is no reason to suppose that it would not 
carry over to other saturating systems, including e.g. inductive provers. 

In the future, we will combine the basic learning cycle and the abstraction 
provided by clause patterns with other learning algorithms. We have already 
investigated various versions of term space mapping (strictly speaking, pattern 
memorization is subsumed as a special case) with good success fSch()()| . We may 
also use folding architecture networks KIK<lblSK<W7 l. a much more powerful, 
but rather slow learning algorithm based on the neural network paradigm. 

We also will try to combine proof experiences generated using a wide variety 
of different search heuristics, and to work with much larger knowledge bases. 
In all of our experiments to date, the performance of heuristics based on pat- 
tern memorization has increases significantly with an increase in stored proof 
experiences. We therefore expect significant synergy effects from this approach. 

Finally, our prototypical implementation can significantly benefit from a 
clean-up and partial redesign of the time-critical sections of the code. 
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Abstract. Association rules are used to investigate large databases. The 
analyst is usually confronted with large lists of such rules and has to find 
the most relevant ones for his purpose. Based on results about knowledge 
representation within the theoretical framework of Formal Concept Anal- 
ysis, we present relatively small bases for association rules from which all 
rules can be deduced. We also provide algorithms for their calculationQ 



1 Introduction and Motivation 



One of the core tasks of Knowledge Diseovery in Databases (KDD) is the mining 
of association rules (conditional implications). Assoeiation rules are statements 
of the type ‘67 % of the customers buying cereals and sugar also buy milk (where 
7% of all customers buy all three items)’. The task of mining association rules is 
to determine all rules whose confidenees (67 % in the example) and supports (7 % 
in the example) are above user-defined thresholds. Since the problem was stated 
IP, various approaches have been proposed for an increased efficiency of rule 
discovery in very large databases j‘2l7l1 1 13(M31| . However, fully taking advantage 
of exhibited rules means providing capabilities to handle them. The problem 
is especially critical when collected data is highly correlated or dense, like in 
statistical databases El . For instance, when applied to a census dataset of 10,000 
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“Bases de Donnees Avancees”, Bordeaux, France, 1999 |2SIi and of the technical 
report p?7|. 
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objects, each of which characterized by values of 73 attributes, experiments result 
in more then 2,000,000 rules with support and confidence greater than or equal 
90%. Thus the question arises: How can long lists of association rules be reduced 
in size? 

Approaches addressing the described issue provide users with mechanisms 
for filtering rules, for instance by user defined templates \4WZ I j . Boolean 
or SQL-like |25| operators or by introducing further measures of “usefulness” 
0; or they attempt to minimize the number of extracted rules a priori by using 
information about taxonomies I1VI15I34I or by applying statistical measures like 
Pearson’ s correlation or the y^-test m- All these approaches have in common 
that they lose some information. 

Our approach, on the other hand, allows us to significantly reduce the num- 
ber of rules without losing any information. We extract only a subset of all 
association rules, called basis, from which all other rules can be derived. This 
approach is orthogonal to the ones mentioned above and can be combined with 
them. 

We make use of techniques of Formal Concept Analysis (FCA). Formal Con- 
cept Analysis [mTTT| arose as a mathematical theory for the formalization of 
the concept of ‘concept’ in the early SOies and is nowadays considered as an 
AI theory. It has since then grown to a technique for data analysis, information 
retrieval, and knowledge representation with over 200 applications, for analyzing 
flight movements at Frankfurt Airport m, for studying semantics of German 
speech-act verbs m, for examining the medical nomenclature system SNOMED 
for IT-security management 0, and for database marketing |TB]. FCA pro- 
vides a framework for KDD, especially for conceptual clustering and association 
rules. A broad discussion of the role of Formal Concept Analysis in data analysis, 
decision support, and KDD is provided in m and m- 

We use results of Duquenne and Guigues cf. also HSl) and Luxenburger 
Eazsi- The former have studied bases (i. e., minimal non-redundant sets of rules 
from which all other rules can be derived) for association rules with 100 % con- 
fidence, and the latter association rules with less than 100 % confidence, but 
neither of them considered the support of the rules. We adopt their results to 
association rules (where both the support and the confidence are considered) 
and provide algorithms for computing the new bases by using iceberg concept 
lattices Ei- We follow an approach in two steps. In the first step, we compute 
the iceberg concept lattice for the given parameters. It consists of all FCA con- 
cepts whose extents exceed the user-defined minimum support. In the second 
step, we derive the bases for the association rules. In this paper, we focus on 
the second step. For the first step, we refer to the Pascal |B| and Titanic jSH] 
algorithms. 

This two-step approach has two advantages compared to the classical two- 
step approach j2j (which computes all frequent itemsets as intermediate result, 
and not only those which are intents of frequent FCA concepts): 

1. It allows to determine bases for non-redundant association rules and thus to 

prune redundancy. 
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2. It speeds up the computation, especially for strongly correlated data or when 

the minimum support is low. 

In 1^, we have presented another pair of bases, which provide rules with 
minimal antecedents and maximal consequents. Compared to the results pre- 
sented here, they have the disadvantage of a higher total number of rules. For 
the approximate rules, M. Zaki has presented similar results in 10. However, 
he does not provide inference rules for support and confidence derivation, does 
not discuss minimality of his results, and does not provide algorithms for the 
computation of the bases. 

The remainder of this paper is as follows. After having recalled some basic 
definitions in Sectional we introduce two bases for association rules in Sectional 
the Duquenne-Guigues basis for exact association rules (i. e., for all rules with a 
100% confidence), and the Luxenhurger basis for approximate association rules 
(i.e., with a confidence < 100%). In Section 2] algorithms are given which com- 
pute the two bases. We conclude the paper with the presentation of experimental 
results (Section 0 and a discussion of future work (Section EJ. 

2 Formal Concept Analysis and the Association Rnle 
Framework 

In this section, we briefly recall the basic notions of Formal Concept Analysis 
priTTil and the association rule problem For a more extensive introduction 
into Formal Concept Analysis refer to [E|. 

Definition 1. A formal context is a triple K := {G,M,R) where G and M are 
sets and R C G x M is a binary relation. A data mining context for dataset^ is 
a formal context where G and M are finite sets. Rs elements are called objects 
and items, respectively, (o, i) € R is read as “object o is related to item i ” . 

For O C G, we define f{0) := {z S M | Vo G O: (o, i) G R\; and for I C M, 
we define dually g{I) := {o G G | Vz G /: (o, z) G R}. A formal concept is a pair 
{0,1) G fP(G) X *P(M) with f{0) = I and g{I) = O. O is called extent and 
I is called intent of the concept. The set of all concepts of a formal context K 
together with the partial order (Oi,/i) < { 02 , 12 ) Oi C O 2 (<^=^ I 2 C Ii) 
is a complete lattice, called concept lattice of K. 

In this setting, we call each subset of M also itemset, and each intent I 
also closed itemset (since it satisfies the equation I = f{g{I))). For two closed 
itemsets I\ and I 2 , we note I\ < I 2 if Ii C I 2 and if there does not exist a closed 
itemset I 3 with /i C /s C aB 

In the following, we will use the composed function h := f o g: fp(M) -G iP(M) 
which is a closure operator on M (i.e., it is extensive, monotonous, and idem- 
potent). The related closure system (i.e., the set of all / C M with h{I) = I) is 
exactly the set of the intents of all concepts of the context. 

We write A C T if and only if A C T and A / T. 
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Definition 2. Let I C M, and let minsupp, mineonf G [0,1]. The support 
count of the itemset I in 'K is supp{I) := I is said to be frequent if 

supp{I) >minsupp. The set of all frequent itemsets of a context is denoted FI. 

An association rule is a pair of itemsets Ii and I 2 , denoted /i — >■ J 2 , where 
I 2 ^ 0. Ii and I 2 are called antecedent and consequent of the rule, respectively. 
The support and confidence of an association rule r := I\ ^ I 2 are defined 
as follows: supp{r) := confer) := conf(r)=l, then r is 

called exact association rule (or implication^, otherwise r is called approximate 
association rule. 

An association rule r holds in the context if supp{r) > minsupp and confer) > 
mineonf The set of all association rules holding in K for given minsupp and 
mineonf is denoted AR. 



Remark 1. The definition of association rules often includes the additional con- 
dition /i n /2 = 0. This condition helps pruning rules which are obviously redun- 
dant, as /i I 2 and I\ — > l 2 \I\ have same support and same confidence. In this 
paper, we omit the condition, in order to simplify definitions. When discussing 
the algorithms, however, we will use the condition since it saves memory. 

The association rule framework has first been formulated in terms of Formal 
Concept Analysis independently in UBi, Eg, and 021 ■ EHl provided also the first 
algorithm (named Close) based on this approach. 

Example 1. An example data mining context K consisting of five objects (iden- 
tified by their OID) and five items is given in Figured together with its concept 
lattice. The association rules holding for minsupp = 0.4 and mineonf = 1/2 are 
shown in the lower table. 

In the line diagram, the name of an object g is always attached to the node 
representing the smallest concept with g in its extent; dually, the name of an 
attribute m is always attached to the node representing the largest concept with 
m in its intent. This allows us to read the context relation from the diagram 
because an object g has an attribute m if and only if there is an ascending 
path from the node labeled by g to the node labeled by m. The extent of a 
concept consists of all objects whose labels are below in the diagram, and the 
intent consists of all attributes attached to concepts above in the hierarchy. For 
example, the concept labeled by ‘A’ has {1, 3, 5} as extent, and {A, C} as intent. 

An example for an exact rule (implication) which holds in the context is 
{A, B} — )> {C,E}. It can also be read directly in the line diagram: the largest 
concept having both A and B in its intent is the one labeled by 3 and 5, and 
it is below or equal to (here the latter is the case) the largest concept having 
both C and E in its intent. This implication can be derived from two simpler 
implications, namely {A} — >■ {C} and {B} — {E}. The aim of the frequent 
Duquenne-Guigues-basis which we introduce in the next section is to provide 
only a minimal, non-redundant set of implications to the user. That basis will 
include the two simpler implications. 
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Fig. 1. The example data mining context K and its concept lattice. The table shows 
all association rules that hold in K for minsupp = 0.4 and minconf = Xj’l. 



At the end of this section, we give some simple facts about association rules. 
We will refer to them later as derivation rules. 

Lemma 1. Rules 1 and 2 hold for (j) S {conf, supp}. 

1. (f{X ^Y) = (j){X ^Y\Z), for allZ<ZXCM,YCM. 

2. 4>{h{X) h{Y)) = <f){X Y), for all X,Y CM. 

3. conf(A — >■ F) = p A conf(F Z) = q => conf(A Z) = p ■ q, 

for all frequent eoncept intents X CY C Z . 

3’. supp(A Z) = supp(F — >• Z), for all X,Y C Z. 

4- conf(A — >■ X) = 1, for all X C M. 



Proof. The proofs for the confidence are given in 1221. 

1. supp(A — )> F) = supp(A ~^Y\Z) follows from AUF = X {Y \ Z) and 
the definition of the support count. 

2. supp(h(A) — >• h{Y)) = supp(A — t> F) follows from g{h{X) U h{Y)) = 
g{h{X))Cg{h{Y)) = g{f {g{X))) C g{f {g{Y))) = g{X)Cg{Y) = g{XCY) by 
using the facts g{f{g{X))) = g{X) and g{X U F) = g{X) fl g{Y) provided 

in HS|. 

3’. supp(A ^Z)= ^ = supp(F ^Z) □ 
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Fig. 2. Frequent closed itemsets extracted from K for minsupp = 0.4. 



3 Bases for Association Rules 



In this section, we recall the definition of iceberg concept lattices and show that 
one can derive all frequent itemsets and association rules from them. Then we 
characterize the Duquenne-Guigues basis for exact association rules and the 
Luxenburger basis for approximate association rules and show that all other 
association rules can be derived from these two bases. 

Definition 3. A concept {0,1) is called frequent concept if supp{I) (= |^) > 
minsupp. The set of all frequent concepts is called iceberg concept lattice. An 
itemset I is called frequent intent (or frequent closed itemset^ if it is intent of 
a frequent concept (i. e., its support is at least minsupp). The set of all frequent 
closed itemsets in K is denoted FC. 



Example 2. The frequent closed itemsets in the context K for minsupp— 0.4 are 
presented in Figure ^together with the semi-lattice of all frequent concepts. Both 
the table and the diagram provide the same information. Note that, in general, 
the set of frequent concepts is not a lattice, but only a semi-lattice (consider e. g. 
minsupp = 0.5 in the example). 



Lemma 2 (|3ll). i) The support of an itemset I is equal to the support of the 
smallest closed itemset containing I, i. e., supp{I) = supp{h{I)). 

ii) The set of maximal frequent itemsets {/ S FI \ fil'GFT.I C I'} is iden- 
tical to the set of maximal frequent closed itemsets {/ S FC \ (U'gFC: I C I'}. 

The next theorem shows that the set of frequent closed itemsets with their 
support is a small collection of frequent itemsets from which all frequent itemsets 
with their support and all association rules can be derived. I. e., it is a condensed 
representation in the sense of Mannila and Toivonen m- This theorem follows 
from Lemma El 

Theorem 1. All frequent itemsets and their support, as well as all association 
rules holding in the dataset, their support, and their confidence can be derived 
from the set FC of frequent closed itemsets with their support. 
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3.1 Duquenne-Guigues Basis for Exact Association Rules 

Next we present the Duquenne-Guigues basis for exact association rules. It is 
based on the following closure operator. 

Theorem 2. The set FIU{M} is a closure system on M , and its related closure 
operator ~ is given by I := h{I) if supp{I) >minsupp and I := M else. 

Proof. The set of all frequent itemsets together with M is a closure system, as 
well as the set of all concept intents. Hence FIU{M} is, as intersection of those 
two closure systems, also a closure system. The proof of the fact that ~ is the 
corresponding closure operator is straightforward. □ 

Our basis adopts the results of m to the association rule framework, where 
additionally the support of the rules has to be considered. 

Definition 4. An itemset I C M in K is a “-pseudo-closed itemset (or pseudo- 
closed itemset for short) 0 «/ / / and for all pseudo-closed itemsets J with 

J G I, we have J G I . The set of all frequent pseudo-closed itemsets in K is 
denoted FP, the set of all infrequent pseudo-closed itemsets is denoted IP. In 
the (unlikely) case that all itemsets are frequent except the whole set M, we let 
IP := {M} (in order to distinguish this situation from the one where all itemsets 
are frequent). 

The Duquenne-Guigues basis for exact association rules ( or frequent Du- 
quenne-Guigues basis^ is defined as the tuple FDG := {L,IP) with C := {I\ — >■ 
h{Ii) I Ji S FP) and IP as defined above. 



Theorem 3. From the Duquenne-Guigues basis for exact association rules one 
can derive all exact association rules holding in the dataset by applying the fol- 
lowing rules. Rules ii) to iv) can be applied to C as long as they do not contra- 
dict (f). 

i) If there exists I S IP with I Q IiG I 2 , then I\ — >■ I 2 does not hold (because 
its support is too low). 

ii) X ^ X holds. 

Hi) If X ^ Z holds, then also X UY Z. 

iv) If X ^ Y and Y \J Z ^ W hold, then also XiJ Z ^ W . 

Proof. We only sketch the proof here, which applies results of m (see also m)- 
One has to check that C U {I^M \ I S IP) is the Duquenne-Guigues-basis (in 
the traditional sense, cf. to fl2ll5l ~l of the closure system FGG )M). Rule (f) 
reflects the implications of the form I — >■ M. □ 

The Duquenne-Guigues basis for exact association rules is not only minimal with 
respect to set inclusion, but also minimal with respect to the number of rules in 
C plus the number of elements in IP, since there can be no complete set with 

^ We do not consider psendo-closed itemsets with respect to other closure operators 
than “ (especially not with respect to h) in this paper. 
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fewer rules than there are frequent pseudo-closed itemsets Observe that, 

although it is possible to derive all exact association rules from the Duquenne- 
Guigues basis, it is not possible in general to determine their support^ 

Example 3. The set of frequent pseudo-closed itemsets of K for mmsupp=0A 
and minconf =1/2 is FP = {{A}, {B}, {if}}, the set of infrequent pseudo-closed 
itemsets is IP = {{ii}}. The Duquenne-Guigues basis is presented in Figure 0 

3.2 Luxenburger Basis for Approximate Association Rules 

In |22i23j . M. Luxenburger discusses bases for partial implications. A partial 
implication is an association rule where the support is not considered. He ob- 
served that it is sufficient to consider rules between concept intents only, since 
conf(A — >■ y) = conf(/i(A) — >■ h{Y)). However, his derivation process does not 
only consist of deduction rules which can be applied in a straightforward manner, 
but it requires to solve a system of linear equations. 

In the KDD process, however, we have to consider the trade-off between the 
amount of information presented to the user, and the degree of its explicitness. 
The appearance of the system of linear equations indicates that Luxenburger’s 
results are in favor for a minimal amount of information presented, and against 
a higher degree of explicitness. As one of the requirements to KDD is that the 
results should be “ultimately understandable” IS], we want to emphasize more 
on the explicitness of the results. Therefore we restrict now the expressiveness 
of the derivation process. This forces the association rules presented to the user 
to be more explicitly 

In the sequel, we consider the derivation rules given in Lemma 0 We present 
a basis for the approximate association rules for these derivation rules. 

Definition 5. The Luxenburger basis for approximate association rules is given 
hy LB := {{r,supp{r), conf{r)) \ r = Ii ^ h, h,h e FC, h < h, confer) > 
minconf, supp{l 2 ) > minsupp} . 

Theorem 4. From the Luxenburger basis LB for approximate association rules 
one can derive all association rules holding in the dataset together with their 
support and their confidence by using the rules given in Lemma 0 Furthermore, 
LB is minimal (with respect to set inclusion) with this property. 

Proof. In order to determine if an association rule r := I ^ J holds in a con- 
text (and for determining its support and its confidence) one can consider the 
rule /' — >■ J' with /' := h{I) and J' := h{I U J) which has (by Rules 1 & 
2) the same support and the same confidence. If I' = J' , then conf{r) = 1 
and supp{r) =supp(/'). If I' yf J' , then exists a path of approximate rules. 

Even if the support for all rules in the basis is known. With the knowledge about all 
frequent closed itemsets and their support however, this is possible (see Theorem QJ. 
® Note that in the KDD setting the user will never actually perform longer series of 
inference steps. 
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Luxenburger basis 

Approximate rule Support Confidence 



BCE A 


0.4 


2/3 


AC BE 


0.4 


2/3 


BE ->■ C 


0.6 


3/4 


C ->■ BE 


0.6 


3/4 


C ->■ A 


0.6 


3/4 


BE 


0.8 


4/5 


C 


0.8 


4/5 



4 / 5 ^ •4/5 
B, E C 3/4 

A 

3/4 ^.6 \ 0.6 



Duquenne-Guigues basis 


C 


(Support) 


A C 


0.6 


B ->■ E 


0.8 


E B 


0.8 






Fig. 3. Duquenne-Guigues and Luxenburger bases for minsupp=0A and minconf =1/2. 



i. e., there are frequent closed itemsets with li — >■ 7^+1 £ LB and 

I' = I\ and In = J' ■ Support and confidence of r can now be determined by 
supp{r) = supp{In) (Rule 3’) and conf{r) = LI'/^.^ conf{Ii — >• li+i) (Rule 3). 

Now we show the minimality of LB. Let r := I ^ J G LB. We show that 
the confidence of r cannot be derived from LB \ {r} by applying the rules of 
Lemma 2. Rule 1 cannot be applied forward since J already contains I. It cannot 
be applied backward because of / ^ J. Rule 2 cannot be applied forward since 
I = h{I) and J = h{J). It cannot be applied backward as LB contains only rules 
with closed antecedent and closed consequent. Rule 3 cannot be applied since 
there is no AT C M with / — >■ AT £ LB \ {r} and K ^ J G LB \ {r} (because of 
I < J). Rule 4 cannot be applied since I ^ J. □ 

Remark 2. A basis in the sense of 123 ] is a maximal spanning tree of our basis 
(when considered as undirected graph) containing at most one rule with M as 
conclusion 0 

Example 4- The Luxenburger basis for approximate association rules of K for 
minsupp=Q.A and minconf =1/2 is also presented in Figure 0 It provides the 
same information as the list in Figure 0 but in a more condensed form. The 
Luxenburger basis is visualized in the line diagram in Figure 0 From its defini- 
tion it is clear, that each approximate rule in the basis corresponds to (at most)0 
one edge in the diagram. The edge is labeled by the confidence of the rule (as a 
fraction), and its lower vertice is labeled by its support (as a rational). Implica- 
tions (exact rules) can be read in the diagram in the standard way described in 
Section 0 

As example for the proof of Theorem 0 let us check if {B} — >■ {A} holds 
in the context for minsupp=Q.A and minconf =1/2. We have I := {B} and 

® The second condition is negligible in KDD, as it follows directly from minsupp > 0 %. 
^ In general, there may be edges which do not represent any rule in the Luxenburger 
basis. Consider for instance minconf =7 /lO. In this case, the two lowest edges would 
not stand for a valid approximate rule, and would hence not be labelled. 
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J := {A}. The smallest frequent closed itemset containing B is /' := {B,E} 
and the smallest one containing A and B is J' := {A, B, C, E}. In the diagram, 
/' and J' are always represented by the largest concepts which are below all 
attributes in / and lU J, resp. Between the two concepts we find the path Ii := 

I 2 '■= {B, C, E}, and I 3 := J' . Hence supp{B — ?> A) =supp{J') = 0.4 > minsupp 
and conf{B — )> H) = conf{Ii — >■ l 2 )'Conf{l 2 — t I 3 ) = 3/4 • 2/3 = 2/4 > minconf, 
which means that the rule holds. 

4 Algorithms for Computing the Bases 

The algorithms presented in this paper assume that the iceberg concept lattice is 
already computed. There are several algorithms for computing iceberg concept 
lattices: the algorithm Close for strongly correlated data the algorithm A- 
Close for weakly correlated data |3I|, the algorithms CLOSET [S2|, ChARM j^ . 
and Titanic PHisni. The algorithm Pascal computes all (closed and non- 
closed) frequent itemsets, but can be upgraded to determine also their closures 
with almost no additional computation time by using the fact that, for I C M, 

h{I) = J U {to G M\I \ supp{I) = supp{I U {to})} . 

When the iceberg concept lattice is computed, then the Duquenne-Guigues basis 
and finally the Luxenburger basis are computed. 



4.1 Generating the Duquenne-Gnigues Basis for Exact Association 
Rules with Gen-FDG 

In this section, we present an algorithm that determines the Duquenne-Guigues 
basis using the iceberg concept lattice. This algorithm (which has not been 
presented before) implements Definition 0 As it needs to know the closure of 
frequent itemsets, it is best applied after an algorithm like Pascal with the 
modification mentioned above, ChARM, or CLOSET. 

The pseudo-code is given in Algorithm E The algorithm takes as input the 
sets Eli, containing the frequent itemsets and their support, and the 

sets ECi, 0<i<k, containing the frequent closed itemsets and their support. 
It first computes the frequent pseudo-closed itemsets iteratively (steps 2 to 17). 
In steps 2 and 3, the empty set is examined. (It must be either a closed or a 
pseudo-closed itemset by definition.) The loop from step 4 to 17 is a direct imple- 
mentation of Definition 01 for the frequent pseudo-closed itemsets. The frequent 
pseudo-closed Aitemsets, their closure and their support are stored in F Pi. They 
are used to generate the set £ of implications of the Duquenne-Guigues basis for 
exact association rules DG (step 18). 

The set of infrequent pseudo-closed itemsets is determined in steps 19 to 21 
using the function £*-closure (Algorithmic). This function uses the fact that, for 
a given closure system, the set of all closed or pseudo-closed sets forms again a 
closure system d. Hence one can generate all closed sets and pseudo-closed sets 
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Algorithm 1 Generating the Duquenne-Guigues basis with Gen-FDG. 

1 ) ^ {}; 

2) if {FCo = {}) then FPq ^ 0; 

3) elseFPo^{}; 

4) for (i •«— 1; j < fc; i++) do begin 

5) FPi^ FIi\FCi-, 

6) forall L £ FPi do begin 

7) pseudo £- true-, 

8) forall P £ FPj with j < i do begin 

9) if (P C L) and (P. closure g L) 

10) then do begin 

11) pseudo false-, 

12) FPi £- FPi \ {L}-, 

13) endif 

14) end 

15) if {pseudo = true) then L. closure ■£- minc({G £ FCj^i \ L C G}); 

16) end 

17) end 

18) forall P € IJHi FP, do C £- C U {P ^ (P.closure\P)}; 

19) 7P^0; 

20) forall L £ MI do IP -t- IP U {£*-closure(7)}; 

21) IP £- mine 7P; 



iteratively by using the corresponding closure operator £*-closure(Z) := IJ^q Zi 
with Zq := Z and := Z, U \J{Y\X^Y £ C,X C Zi} The set £ of 

implications has the form £ = {Xi — >• Fi, . . . , A„ — >• Yn}. 



4.2 Generating the Luxenburger Basis for Approximate Association 
Rules with Gen-LB 

The pseudo-code generating the Luxenburger basis for approximate association 
rules is presented in Algorithm]^ The algorithm takes as input the sets FCi, 
0<i<k, containing the frequent closed itemsets and their support. The output 
of the algorithm is the Luxenburger basis for approximate association rules LB. 
The algorithm iteratively considers all frequent closed itemsets L £ FCi for 
2 < i < k. It determines which frequent closed itemsets L' e [Jj<iFCj are 
covered by L and generates association rules of the form £' — )► L\ L' that 
have sufficient confidence. During the iteration, each itemset L in FCi is 
considered (steps 3 to 13). For each set FCj, l<j<i, a set Sj containing all 
frequent closed j-itemsets in FCj that are subsets of L is created (step 4). Then, 
all these subsets of L are considered in decreasing order of their sizes (steps 5 
to 12). For each of these subsets £' £ Sj, the confidence of the approximate 
association rule r :=£'—?>£ \ £' is computed (step 7). If the confidence of r is 
sufficient, r is inserted into LB (step 9) and all subsets L” of £' are removed 
from Si, for I < j (step 10). At the end of the algorithm, the set LB contains 
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Algorithm 2 Function £*-closure reads X and returns its £*-closure C*{X). 
1 ) 

2 ) for (i 1 ; i = n; i + +) do i.used<— false; 

3) repeat 

4) changed false; 

5) If Subsets(7P, y) 7 ^ 0 then begin Y •<— M; changed- 4 — true end 

6 ) else for (i •<— 1 ; i < n; i + +) do 

7) if Ai C y then begin Y •<— y U y ; changed- 4 — true end 

8 ) until not changed; 

9) return Y 



Algorithm 3 Generating the Luxenburger basis with Gen-LB. 

1 ) LB^{}; 

2) for {i 2; i <k; i++) do begin 

3) forall L £ FCi do begin 

4) for {j •<— 0; J < i;j + +) do Sj ■£- Subsets(PG, , L); 

5) for (i ■<— i — 1; J > 1; i ) do begin 

6) forall L' £ Sj do begin 

7) conf <— L. support / L'. support; 

8) if {conf > minconf) 

9) then LB ■£- LB U {{L' — >■ (L \ L'), L. support, conf)}; 

10) for {I j;l> l;l ) do Si Si\ Subsets(S'i, L'); 

11) end 

12) end 

13) end 

14) end 



all rules of the Luxenburger basis for approximate association rules. The proof 
of the correctness of the algorithm is given in m 

5 Experimental Results 

We have preformed several experiments on synthetic and real data. The char- 
acteristics of the datasets used in the experiments are given in Table n These 
datasets are the T10I4D100K synthetic dataset that mimics market basket datap 
the G20D10K and the G73D10K census datasets from the PUMS sample filell 
and the Mushrooms dataset describing mushroom characteristics 0 In all ex- 
periments, we attempted to choose significant minimum support and confidence 
threshold values. We varied these thresholds and, for each couple of values, we 
analyzed rules extracted in the bases. 

Number of Rules. Table Elcompares the size of the Duquenne-Guigues basis for 
exact rules with the number of all exact association rules, and the size of the 

® http: / /www. almaden.ibm.com /cs/ quest / syndata.html 
® ftp://ftp2.cc.ukans.edu/pub/ippr/census/pums/pums90ks.zip 
^°ftp://ftp.ics.uci.edu/'cmerz/mldb.tar.Z 
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Table 1. Datasets. 



Name 


Number of objects 


Average size of objects 


Number of items 


T10I4D100K 


100,000 


10 


1,000 


Mushrooms 


8,416 


23 


127 


C20D10K 


10,000 


20 


386 


C73D10K 


10,000 


73 


2,177 



Table 2. Number of exact and approximate association rnles compared with the num- 
ber of rules in the Dnquenne-Guignes and Lnxenburger bases. 



Dataset 


Exact 


D.-G. 




Approximate 


Luxenburger 


(Minsupp) 


rules 


basis 


Minconf 


rules 


basis 








90% 


16,269 


3,511 


T10I4D100K 


0 


0 


70% 


20,419 


4,004 


(0.5%) 






50% 


21,686 


4,191 








30% 


22,952 


4,519 








90% 


12,911 


563 


Mushrooms 


7,476 


69 


70% 


37,671 


968 


(30%) 






50% 


56,703 


1,169 








30% 


71,412 


1,260 








90% 


36,012 


1,379 


C20D10K 


2,277 


11 


70% 


89,601 


1,948 


(50%) 






50% 


116,791 


1,948 








30% 


116,791 


1,948 








95% 


1,606,726 


4,052 


C73D10K 


52,035 


15 


90% 


2,053,896 


4,089 


(90%) 






85% 


2,053,936 


4,089 








80% 


2,053,936 


4,089 



Luxenburger basis for approximate rules with the number of all approximate 
rules. In the case of weakly correlated data (T10I4D100K), no exact rule is 
generated. The reason is that in such data all frequent itemsets are frequent 
closed itemsets. However, the Luxenburger basis is relatively small compared to 
the number of all rules, since only immediate neighbors with respect to the subset 
order (and not arbitrary pairs of sets) are considered. In the case of strongly 
correlated data (Mushrooms, C20D10K and C73D10K), the ratio between the 
size of the bases to the number of all rules which hold is much smaller than in 
the weekly correlated case, because here only few of the frequent itemsets are 
closed and have to be considered. 



Relative Performance. Our experiments also show that in all cases the execution 
time of Gen-FDG and Gen-LB are insignificantly small compared to those of the 
computation of the iceberg concept lattice, since both algorithms need not access 
the database. We can conclude that without additional computation time (com- 
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pared to other approaches, like e. g. Apriori) our approach not only computes 
all frequent closed itemsets but also the two bases described in Section El 

6 Outlook 

In this paper, we introduced bases which significantly reduce the number of asso- 
ciation rules presented to the user without losing any information; and provided 
algorithms for computing them. This work is currently extended in different 
directions: Integrating reduction methods. Templates, as defined in jaST] , can 
directly be used for extracting all association rules matching some user specified 
patterns from the bases. Information in taxonomies and ontologies associated 
with the dataset can also be integrated in the process as proposed in nnn 
for extracting bases for generalized (multi-level) association rules. Integrating 
item constraints mm and statistical measures m in the generation of bases 
requires further work. 

Integration of association rule visualization in Conceptual Information Sys- 
tems. Using the technique of conceptual scaling. Conceptual Information Systems 
present the information contained in large databases to the user in conceptual 
hierarchies of a manageable size |40l‘lfillSj . We work on exploiting this visual- 
ization techniques for presenting also association rules to the user. 

Supporting the creation of new concepts in Description Logics. In Descrip- 
tion Logics, currently approaches are discussed to support the domain expert in 
creating new concepts which regroup more specific similar concepts |3|. Those 
approaches extend the partial order of the concepts in the terminology to a 
lattice and suggest new concepts to the user. Since the more specific concepts 
are often defined incoherently, the user is often interested in only approximate 
relationships between those concepts, and on a general level only. It is planned 
to adapt the bases and the algorithms presented in this paper to that task. 
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Abstract. This paper extends previous work on the representation and analysis 
of Java programs for diagnosis in a new direction by providing a description and 
analysis of the issues arising from handling object references in dependency-based 
models of Java programs. We empirically compare dependency-based models 
with a value-based model using a set of example programs in terms of required 
user interaction (questions put to the user) and examine and incorporate specific 
interesting error categories. Apart from being based on experience with an actual 
implementation of the various models, the model extensions and analysis deal with 
aliasing, an issue that the programming language community has been examining 
for a long time, and that is also crucial to object-orientedness. 



1 Introduction 

Detecting, locating, and repairing faults in software is a difficult and time consuming task. 
Detecting an incorrect behavior of a given program is done by using testing techniques 
(e.g., O) or formal verification methods, e.g., model checking Whereas much effort 
has been made on test theory, test methodology, and algorithms for automatic test-case 
generation, somewhat less work has been published on locating and repairing software 
faults. Because debugging is not only performed in the implementation and test phases 
of a project, but also in maintenance, saving debugging time naturally results in saving 
time and money over the whole product life cycle. Especially in maintenance, where the 
original developers may no longer be involved, debugging is very costly. An automated 
debugger for locating and fixing faults can help in such a situation. 

Automatic debugging approaches introduced in the past include program slicing E2 
E3, algorithmic debugging Ea, dependency-based techniques iHTnrmi , probability- 
based methods and others. An overview of automatic debugging techniques can be 
found in 0. These traditional approaches are either specific to a programming language, 
use specialized algorithms, or require explicit user-interaction to locate a bug. In order to 
overcome these drawbacks and to improve the results of the abovementioned approaches, 
the use of model-based diagnosis (MBD) for debugging was suggested 0| . Model-based 
diagnosis f 1 provides a general theory for diagnosis that has sucessfully been applied 
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to various engineering areas. Apart from the diagnosis of technical systems, e.g., El 
El, MBD has been used for knowledge bases lEOll and other less technical domains, 
e.g., in ecology 1TUI . 

In this paper we build on the results obtained by the MBD community, especially as its 
application to the localisation of software bugs is concerned ( [01 and more relevantly |3 
1211 1. Section 12 gives a brief description of the basics of MBD and shows how it can be 
used for fault localisation in programs. Section [^presents some general thoughts about 
the modeling of Java programs and lists a sample program, which will be used throughout 
this paper to describe various model properties. Section 0 describes how a functional 
dependency model (FDM) of Java programs can automatically be derived from the 
program’s source code. Furthermore, the transformation into a simplified functional- 
dependency model and the handling of object references (and therefore aliasing) are 
discussed. SectionElshows how semantic information can be incorporated into the model 
by introducing a value-based model (VBM). Such a model allows for better results at 
the cost of an increased debugging time. Section 0 describes how the two models can 
be used to actually locate faults in programs. Furthermore, it presents empirical results 
produced by the j ade prototype debugger. The j ade debugging tool is a model-based 
debugger for Java programs featuring both, the use of functional dependency and value- 
based models. The diagnosis engine used by jade is based on Reiter’s algorithm IfTRl . 
The j ade user interface is designed to guide the user through the code in an optimal 
fashion, with optimality defined as minimizing the user interaction. An analysis of the 
debugging potentials of our approach and some future extentions conclude this work. 

2 Model-Based Diagnosis and Debugging 

The model-based approach is based on the notion of providing a representation of the 
correct behavior of a technical system. By describing the structure of a system and the 
function of its components, it is possible to ask for the reasons why the desired behavior 
was not achieved. In the diagnosis community, the model-based approach has achieved 
wide recognition due to the following advantages: 

- once an adequate model has been developed for a particular domain, it can be used 
to diagnose different actual systems of that domain 

- the model can be used to search for single or multiple faults in the system without 
alteration 

- different diagnosis algorithms can be used for a given model 

- the existence of a clear formal basis forjudging and computing diagnoses 

Using the standard consistency-based view as dehned by Reiter fTK||. a diagnosis 
system can be formally seen as a tuple {SD, COMP) where SD is a logical theory 
sentence modeling the behavior of the given system (in our case the program to be 
debugged), and COM P a set of components, i.e., statements or expressions. A diagnosis 
system together with a set of observations OBS, i.e., a test-case, forms a diagnosis 
problem. A diagnosis A, i.e., a bug candidate, is a subset of COMP, with the property 
that the assumption that all statements (expressions) in A are incorrect, and the rest of the 
statements (expressions) is correct, should be consistent with SD and OBS. Formally, 
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Z\ is a diagnosis iff SD U OBS U {-^AB{C)\C € COMP \ A} U {AB{C)\C € 
A} is consistent. A component not working as expected, i.e., a statement (expression) 
containing a bug, is represented by the predicate AB{C). 

The basis for this is that an incorrect output value (where the incorrectness can be 
observed directly or derived from observations of other signals) cannot be produced by a 
correctly functioning component with correct inputs. Therefore, to make a system with 
observed incorrect behavior consistent with the description and avoid a contradiction, 
some subset of its components must be assumed to work incorrectly. In practical terms, 
one is interested in finding minimal diagnoses, i.e., a minimal set of components whose 
malfunction explains the misbehavior of the system (otherwise, one could explain every 
error by simply assuming every component to be malfunctioning). Basic properties of 
the approach as well as algorithms for efficient computation of diagnoses are described 
in Ca. 

The principles of model-based debugging are depicted in Figure ID The program, in 
our case written in Java, is compiled into an internal representation. From this represen- 
tation (together with a set of model fragments) a converter computes logical models for 
diagnosis. Model fragments represent a logical description of parts of a model, e.g., the 
behavior description of functions. Such knowledge has to be derived from the program- 
ming language semantics. For Java for example, a value-based model requires the model 
fragments of all basic functions and types of statements, e.g., for arithmetic functions, 
behaviors (e.g., Nab{C) — >■ out{C) = ini{C) + iri2{C) for the + operator) must be 
defined. The predicate Nab stands for not abnormal, saying that a function or state- 
ment is not responsible for an incorrect behavior. After building the model, which is 
done automatically, the model together with the specihed behavior of the program, e.g., 
test-cases, is used by the diagnosis engine for finding bug candidates. The candidates 
can be further discriminated by adding additional knowledge, i.e., values of variables 
at specihc locations within the program. The selection of the variable and location is 
done by a measurement selection algorithm. The information about the value must be 
delivered by the user (or another oracle). The remaining candidates provide a link back 
to the original source code. 




Fig. 1. MBD in software debugging 
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class Point { 
int x; 
int y; 

Point(int x, int y) { 



1. pi = new Point(0,0); 

2. p2 = new Point(2,3); 

3. pl.x = 1; 

4. pl.y = 2; 

5. p2 = pl.plus(p2); } } 



public static void test() { 



Point pi, p2; 



1. this.x = x; 



2. this.y = y; } 

Point plus(Point p) { 



1. return new Point(x+p.x, y+p.y); } 



Fig. 2. Example program Point.java 



3 Modeling for Debugging 

To demonstrate the modeling process and the properties of the resulting models we imple- 
ment a short example program which provides the basic data structures and functionality 
of a 2-dimensional point (see Figured- 

Note that variable declarations are not counted as statements, because no diagnosis 
components can arise from them. We classify the entities in the program in terms of 

- The set C of constants (e.g., 0, 1, 2). Although constants in Java are not objects, 
they can logically be seen as objects with a fixed content of primitive type. Each 
occurrence of such a constant has to be considered separately, but we omit indexes 
for simplicity in the paper. 

- The set C of memory locations which represent the stored state of Java objects. A 
location is a placeholder for objects that would be produced dynamically during 
runtime by constructor calls lO- Their internal state is user defined and alterable 
(in our example 3 locations of type Point are created), but it is important to note that 
locations, which are introduced during static analysis of the functional dependencies, 
do not represent individual objects, but instead are generic representations covering 
all objects that might be assigned to a particular variable occurrence. In the value- 
based model explained later, individual objects are used instead. 

- The set V of variables (including class and instance fields) can be of primitive or 
of reference type. Primitive variables (e.g. x and y above) hold values, reference 
variables (e.g. pi and p2) reference objects, i.e. they point at a particular memory 
location. 

The state of a Java system at any time during execution can be specified by defining 
the three sets of system components given in the list above. Figure 0 shows the state 
of our example program after the execution of all five statements of method test(j. We 
can see the current run time state of three objects (memory locations) given by their 
instance fields which are all assigned to constants. The two variables of reference type, 
pi and p2 reference the first and third location, respectively. The object at location 2 is 
not referenced by any of the system’s variables and will be eliminated by the garbage 
collector. In the dependency-based view, the values of the instance variables of p2 are 
recorded as being derived from the values of those of location 2 (and location 1). 
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(a) Value-based (b) Dependency-based 



Fig. 3. Representing objects 



The following sections show how a Java system as defined above can be modeled for 
debugging using a static functional dependency model that for the first time incorporates 
object references (in SectionEJ and a dynamic value-based model (Sectional). 

4 The Dependency-Based Model 

A dependency model for Java programs was presented in o, defining a functional 
dependency to express the fact that the value of the z-th occurrence of vari- 

able X in the current method depends on the values of the variables (actually, variable 
occurrences) in if their values were used to compute the value stored by Xi (data 
dependency) or if they were used in selection or loop conditions that could have influ- 
enced the computation of that value (control dependency). In this paper, we describe 
two extensions of that model. The central issue in both is the explicit consideration of 
the semantics of object references, allowing a treatment of aliasing situations, which are 
generally excluded in classical dependency research (e.g, m), but are quite relevant in 
Java due to the language’s general reference semantics for object-valued variables. 

4.1 The Detailed Functional Dependency Model 

When creating the detailed functional dependency model (DFDM) of a particular method 
we consecutively transform each statement of the method in question. Functional de- 
pendency components (hereinafter FDs) arise from assignment statements or program 
structures, that themselves include assignments, such as method calls (side-effect FDs), 
selection statements (selection FDs) or loops (loop FDs). Note that there might exist 
more than one FD for one statement, e.g., side-effect, selection, or loop FDs. 

AnFD in the detailed model is of theformfUjriar, DEP), where varis the variable 
occurrence whose value is determined in the particular statement and therefore depends 
on the entities in DEP. var can be of the following form: x if x denotes a class variable 
or a local variable of the currently modeled block; 0::x if x stands for an instance field of 
the receiver of the method; n::x where n> 0 is a location representing a different object 
than the receiver, x denotes an instance field of this particular object. Note that in all 3 
cases X can be of primitive or of reference type. 

DEP is a tuple DEP =< C, V, M, L >, where C C C is a set of constant values 
and y C V is the set of variables influencing var, M is a set of method declarations 
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(i.e., of the methods possibly called in the process of computing the new value for var), 
and L C £ is a set of locations representing those objects that might be referenced by 
var. 

Method test{ ) from the example is modelled by collecting all FDs arising from its 
hve statements. These FDs can be seen in the following report, which is automatically 
generated by the j ade modeling component: 



Jade System Description Global Report 
Statement 1: pl=new Point(0,0) 

FD ( ’l;:y_r { consts(O), vars{), methods(Point::Point(int,mt)), locsQ }) 

FD(’l::x_r { consts(O), vars(), methods(Point::Point(iiit,mt)), locsQ }) 

FD ( ’pl-1’ { constsO, vars(), methods(Pomt::Point(int,int)), locs(I) }) 

Statement 2: p2=new Point(2,3) 

FD ( ’2::y_r { consts(3), vars(), methods(Point::Point(int,mt)), locsQ }) 

FD ( ’2;:x_r { consts(2), varsQ, methods(Point::Point(int,mt)), locsQ }) 

FD ( ’p2.r { constsQ, varsQ, methods(Point::Pomt(int,mt)), locs(2) }) 

Statement 3: pl.x=l 

FD(’l::x_2’ { consts(l), vars(’pl.r), methodsQ, locsQ }) 

Statement 4: pl.y=2 

FD(’l::y_2’ { consts(2), vars(’pl.r), methodsQ, locsQ }) 

Statement 5: p2=pl.plus(p2) 

FD(’3::x_r { constsQ, vars(’pl_r,’l::x_2’,’p2-r,’2::x-r), methods(Point::Point(mt,int),Point::plus(Point)), locsQ }) 

FD ( ’3;:y_r { constsQ, vars(’pl_r,’l::y_2’,’p2-r,’2::y-r), methods(Point::Point(int,int),Point::plus(Point)), locsQ }) 

FD ( ’p2-2’ { constsQ, varsCpl-l’), methods(Point::Point(int,int),Point::plus(Point)), locs(3) }) 



Six of the above FDs are side-effect components and imported into the model of test( ) 
via the calls of Point(int x, int y) and plm(Point p), the others are directly derived from 
the assignment statements in test(). Note that since the FD model is static, its creation 
does not involve or require any information about a particular evaluation trace. 



4.2 A Simplified Functional Dependency Model 

The advantage of the DFDM which records locations and references separately is that 
a broad range of program bugs can be located at statement level, due to the distinction 
being made between memory locations and variables. However, the DFDM is difficult to 
read and understand, and the large size of FDs slows down diagnosis performance. Also, 
the user needs detailed knowledge about the underlying object structure when specifying 
an incorrect variable observation. 

This section therefore introduces a simplified functional dependency model (SFDM), 
which can automatically be derived from the DFDM. It is easier to understand, includes 
only variables on the FDs’ right-hand sides, and makes it easier for the user to specify 
observations. FDs in SFDM again have the simpler structure of m, but due to being 
derived from DFDM, still incorporate the effects of locations. 

The conversion from DFDM to SFDM is obtained by successively simplifying each 
FD of the DFDM. Consider a FD d = {vo, DEP), where DEP =< C,V, M, L >, to 
be converted to a SFD d' = (vo', DEP'), where DEP' is the subset of V that contains 
all local and class variables and only those instance variables, which are defined for the 
method’s owner class, i.e., variable occurrences of the form n::y for n > 0 are deleted. 

For the variable occurrence on the left hand side, assume first that vo is of the form 
X or 0::x (i.e., it is a class or local variable occurrence or instance variable occurrence 
of the analyzed class. Then, vo' = vo. Now, assume that vo is of the form n:.'x referring 
to n G L. Then all (local, class, or instance) variable occurrences y which at the given 
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point within the program possibly reference n are resolved by introducing a new FD of 
the form (y, DEP') (with DEP' as above). 

As a result, FDs for variable occurrences of reference type now no longer simply 
denote that reference. Instead the dependency directly refers to the locations and refer- 
ences representing the local state of the referenced object (since these locations are no 
longer explicitly present in the model). 

The overall set of FDs occurring at run-time can now be covered by C and V' . The 
locations C are no longer explicitly part of the model, but implicitly covered through V'. 

The resulting SFDM for method test() reads as follows: 



FD ( ’pl_r { vars()}) % Statement 1: pl=new Point(0,0); 

FD ( ’p2_r -4= { vars()}) % Statement 2: p2=new Point(2,3); 

FD ( ’pl-2’ { vars(’pl_r)}) % Statement 3: pl.x=l; 

FD ( ’pl_3’ { vars(’pl_2’)}) % Statement 4: pl.y=2; 

FD(’p2_2’4= { vars(’pl_3’,’p2_r)}) % Statement 5: p2=pl.plus(p2); 



These two FD models introduce the notion of aliasing, which describes any situation 
where two variables of reference type refer to two different object structures, which 
in turn have references to the same object in common. Consider the following code 
fragment using two variables pi, p2 of class Point: 

1. p2 = pl; 

2. pl.doubleXValueO; 

3. output(p2.x); 

Assume the call to method doubleXValue() is erroneous, and an incorrect value is 
observed in p2.x (which is the same as pl.x since they refer to the same Point object). 
The model from d would suggest statement 1 as a potential source of the error, but 
would omit statement 2. Both, the DFDM and the SFDM introduce a location I on which 
both pi and p2 depend and express the change caused by the method call in line 2 in 
terms of a change to Lx, thus making the aliasing between the two variables explicit. 



5 The Value-Based Model 

In the value-based diagnosis model both expressions and statements are represented as 
diagnosis components, with the semantics of the expressions and statements described 
in terms of logical sentences. Components are connected if there is a flow of information 
between the corresponding expressions and statements, e.g., if a changes a variable v 
that is accessed in s and there is no assignment to v in between. 

5.1 Description 

FigureEl shows the graphical representation of the model of example program test. The 
source code of line 1 is mapped to four diagnosis components: two components for the 
constants (Cl. 2, Cl. 3), one for the new Point method call (Cl.l), and one (Cl) that 
corresponds to the assignment statement. Instance variable accesses are also represented 
as components (C3 or C4) with two parameters: the variable storing the object, and the 
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name of the instance variable (see line 3 or 4). If variables are used in an expression, 
then they are mapped to a variable access component (see components corresponding 
to line 5). We refer to a component C as "correct" if it is assumed to be correct, i.e., 
^AB{C) holds. 



public static void test() { 



Point pl,p2; 




mp2 



Fig. 4. Graphical representation of the value-based model 



In the following behavior description, we use tuples of the form 
\Id\VariVali\ . . .\VarnValn\\ to represent objects with the unique object iden- 
tifier Id, and the pairs \V avi Vak] that map each instance variable Vari to its value 
Vali. Note that Vak itself can be an object identifier. 

Constants: Correct constants propagate their constant value, e.g., an integer or even 
an object, to the output port. 

Variable access: This has one input, connected to the output of the last assignment 
having the same variable as target, and one output. A correct access propagates the input 
value to the output and known output values to the input. 

Operators: An operator has an input port for each argument and one output. The 
inputs are connected to the components that correspond to subexpressions. The output 
is connected to a statement port, e.g., the expression port of an assignment, or to one 
input of the superexpression. The behavior of an operator is specified by the semantics 
of the corresponding statement. For example the Java and (&&) operator component has 
the following behavior: ~'AB{C) => (mi(C) A m 2 (C) = out{C)) 

Method calls: The behavior of a correct method call is determined by the diagnosis 
model constructed from the method’s source code to propagate the input parameter 
values to output parameters or return value. 

Assignments: If assumed to be correct, assignments v = e to locally declared 
variables propagate the single input value produced by expression e to the single output 
and vice versa. Assignments to instance variables (e.g., pl.x = 3;) have a second 
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input carrying the object. The output port is connected to components which access the 
variable. The correct behavior is to change only the instance variable Vi of the object [Id 
\V ari Vail] . . . [Var^ Vain]] occurring on the input port of the variable. In the back 
propagation step the expected object is propagated from the output to the input port of 
the variable, and the value of the used instance variable is propagated to the output port 
of e. 

Control statements: Loop components have as input connections the variables used 
inside the loop, and as output connections the variables altered by the loop, propagating 
values repeatedly (as needed) across the components of the body of the loop. Selection 
statements propagate values to either the then or else branch depending on condition 
evaluation. 

Diagnoses are computed in the usual consistency-based manner |1 8||: A set Z\ C 
COMP is a diagnosis iff SDUOBSU {^AB{C)]C € COMP\A}U {AB{C)]C G 
Z\} is consistent. AB{C) indicates that a component C is behaving abnormally, and a 
correctly behaving component C is described by ^ab{C) . In general we want to compute 
diagnoses which are subset-minimal. 

To locate a bug in program test, assume that the call test() should compute 
the object [? [x 1] [y 1]] to be stored in variable pi. Instead, test returns the 
object [? [a; 1] [y 2]] for variable pi. The test case produces four diagnoses: 
{AP(Cl.l)}, {AB{C1)}, {AB{C4, 1)}, {AB{CA)} which correspond to statements 
1 and 4. If using a fault model for the new Point method and the assignment statement, 
we can rule out the diagnoses {AB{C1.1)} and {AB)[Cf)} by propagating an empty 
object [? [a; ?] [y ?]] to the output in case of an unknown fault AB. For more details of 
the value-based model see I16II . 

5.2 Refining Conditional Statements 

The value-based model treats conditional statements loosely, i.e., in some cases too many 
diagnoses can be derived from the model. Consider the following program fragment with 
expected value for result being 0: 

1. x=l; 

2. if (x < 1) { 

3. result=l;} 

4. else { result =!;}//! should be 0 ! 

With the original value-based model we derive diagnoses saying that either statement 
1 or statement 2 (including the subblocks) causes the misbehavior. But it is obvious that 
the assignment in line 1 is not a single fault. This effect is due to the missing behavior 
in cases where the correct value of the condition expression of a conditional statement 
is known, which is the case when we assume that line 1 is faulty. A remedy is to extend 
the model of conditional statements: 

1. If there is a variable x such that the then and else block compute different values 
for X and both values are different from the expected value, then raise a contradiction. 

2. If there is a variable x such that then and else block compute the same value v for 
X, propagate v to the output. 

Formally, the new model can be expressed as follows: 
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-'AB{C) Acond{C) = true => {thenx (C) = outx{C)) 

-^AB{C) A cond{C) = false {elsex(C) = outx{C)) 

-^AB(C) A cond{C) = unknown A thenx (C) 7^ outxiC) Aelsex{C) 7^ outxiC) => _L 
->AB(C) AcondiC) = unknown A thenx (C) = elsexiC) {outxiC) = thenx(C)) 

When using this model, line 1 is excluded as a diagnosis candidate. 



6 Results 

In this section we compare the dependency-based and value-based model, first in terms 
of the Java language constructs covered, then in terms of tests carried out with the 
jade debugger on both types of models using actual programs. Note that during all 
tests with FDMs a SFDM was used. Whereas a DFDM could produce more accurate 
results, its handling would be more difficult, especially as far as the exact specification 
of observations is concerned. We therefore use the DFDM as a generic model, which 
represents the basis for the SFDM and in future other possible dependency-based models. 
Finally we analyze some routes for future work. 

Both models currently support classes and instances, static and instance methods and 
variables, method calls, polymorphism and aliasing. The value-based model currently 
does not allow for recursive functions. Both models support sequential code exclusively 
(no processes). 

Since the value-based model expresses the full semantics of the covered language 
features, it can predict values by forward propagation (from inputs to outputs) or back- 
ward propagation (from outputs to inputs) whenever sufficient values are present on 
a component’s connections. During debugging, it allows the user to specify intended 
actual values as observations instead of merely observing whether a value is correct or 
incorrect. Also, the value-based model’s diagnosis granularity is lower and individual 
subexpressions can be identified as incorrect. 



6.1 Using the Models for Debugging 

The diagnosis models were implemented in the context of the j ade debugger and tested 
on a Sun UltraSPARC 11/360. Both models are automatically derived from the source 
code and compiled into a system description which can be represented as a set of logical 
sentences and thus be loaded into a standard theorem proven The program is evaluated 
and the user is then asked to specify expected values (observations) for the connections 
of the system description. This means that when using the SFDM he has to state whether 
the value of a certain variable (directly derived from the observed in/out behaviour of 
the tested method) at a particular position within the source code has the correct value or 
not, and when using the VBM he can specify the intended values. The observations are 
converted into logical sentences and added to the debugger’s system description. The 
user can continue specifying additional variable observations, in order to successively 
reduce the set of possible diagnoses and eventually hnd a single bug location. A mea- 
surement selection algorithm (the jade system uses a slightly modified version of Q) 
automatically selects the measurement point, i.e. variable occurrence, whose evaluation 
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will reduce the number of diagnoses in an ideal way. After a certain number of such 
steps a single statement s containing the bug is found by the debugger. In this case the 
user has the following options: 

1. If s contains subblocks (a selection or loop statement), the user can interactively 
guide the diagnosis process into the condition or one of the subblocks. If s is a loop, the 
user also has to select the first incorrect iteration from a displayed list. When stepping 
into a subblock, information from the enclosing statement s is propagated downwards. 
2. In all other cases the bug has been found at statement or expression level, and in the 
case of a method or constructor call, debugging can continue inside that call. 3. Stop the 
debugging process, continue manually from the current diagnosis candidate list. 



6.2 Empirical Results 

The relative debugging potential of both models was tested on programs demonstrating 
simple variable dependencies (simulating a binary adder, numeric examples), making use 
of control structures (if and while statements), and finally multiple objects and instance 
fields together with linked lists and general processing (a small library application). 

TableElshows the tested method (in which a single error has been installed), the total 
application code length (which determines the model size; esp. in the library example 
most of this code is also called by the buggy function), the number of statements in the 
tested method, the index of the buggy statement within the method (which can directly 
be used to compare the outcomes of the jade debugger tests with "manual" use of a 
debugger where the user steps through the code sequentially until the erroneous line is 
found), and finally the number of user interactions (variable setup, variable observation 
queries, program flow control queries) which are needed to exactly locate the bu£l The 
latter are defined as follows: 

FD shows the total number of user interactions needed to locate a single error at 
statement level using the FD model. 

VI shows the performance of the VBM in statement level debugging. Selection 
statement queries are not needed with the VBM, because the system automatically guides 
the diagnosis process into the faulty subbranch. For each measurement selection query 
only one observation was specified. 

V2 shows the number of interactions needed to locate the faulty expression using the 
VBM. In most cases it took one or two extra steps from statement level to expression 
level. 

In general both debugging strategies perform better than a manual walkthrough. On 
average it takes 1 . 1 additional queries to find the solution at expression level. Therefore 
the VBM usually finds a solution at expression level quicker than the FDM at statement 
level. 

In general the VBM achieves better results than the FDM, mainly due to the ad- 
ditional run-time information used by the VBM debugger, which allows for a more 
effective elimination of wrong diagnoses (see adder and numeric tests). With a grow- 
ing number of variables and more complex dependency structures this effect is likely 

* The average is included for overview purposes; it has no statistical significance since it depends 
on the specific selection of program and error examples. 
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Fig. 5. Debugging results from Java examples 



to increase. If the error appears within an if statement, the VBM clearly outperforms 
the FDM (see if-tests), mainly due to the VBM debugger’s ability to debug the body 
of a selection statement together with its enclosing block, whereas the FDM debugger 
performs a hierarchical diagnosis involving some overhead (control and setup GUIs). 
With loops, both models require extra queries on loop condition correctness (mainly to 
identify the first incorrect iteration) in combination with setup queries for the subblock’s 
variables. This leads to more queries for both, reducing the advantages of the VBM. In 
the numeric examples the apparently high number of queries is somewhat misleading 
since the complex computations automatically produce good focusing and result in iden- 
tifying detailed subexpressions quickly. With the library program both models seem to 
be equally good. Although the VBM uses more information than the SFDM, this does 
not pay off here because of the complex object structure of the application combined 
with the relatively simple operations possible in the example (mostly lookup operations 
concerning lists of books and customers which are highly dependent on the input values 
provided). 

6.3 Discussion 

The j ade debugger utilizes a model-based diagnosis framework to incorporate multiple 
automatically derived models of programs into a standard debugger interface, allowing 
to switch between traditional step-based debugging and between models at the flick of a 
switch and providing iterative error location without the need for external specifications. 
The latter properties are the main distinction compared to formal verification methods 





Comparing Two Models for Software Debugging 



363 



such as model checking, which requires a separate formal specihcation and provides 
counterexamples hut no indication of the error locality. 

Current implementation work includes better runtime for the VBM model, since 
loops with complex object structures such as large lists can result in diagnosis runtimes 
of several minutes. Howevere, most of the examples in the table were diagnosed in the 
1-10 second range. Another goal is better visualization for indicating correctness of 
complex object structures. 

In modeling, the identification and modeling of specihc problem classes can be ex- 
pected to lead to still more effective debugging. The dependency-based representation 
can be expected to scale up well to medium-sized programs (thousands of lines of code). 
(The somewhat simpler dependency-based representation described in |§t| provided ac- 
ceptable performance for programs with hundreds of thousands of lines of code.) 

Faulty location structure. Either two variables point to different locations, i.e., 
objects, but should refer to the same object, or a variable refers to the same object as 
another variable but should point to a different object (possibly with the same content), 
as in this case: 

1. pi = new Point(0,0); 

2. p2 = pl; // Should be pi . copy 0 

3. pl.x=l; // Expected results: pl=(l,0), p2=(0,0) 

When using its simple functional dependency model 

FD(’pl_r <- {varsQ}) % Statement 1: pi = new Point(0,0); 

FD(’p2_r <- {vars(’pl_r)}) % Statement 2: p2 = pi; 

FD(’pl_2’ <- {vars(’pl_r,’p2_r}}, ’p2_2’ <- {vars(’pl_r)}) % Statement 3: pl.x = 1; 



the debugger will list all statements as bug candidates. However, because statement 
3 changes the value of pi and its value is correct at the end of the program, the statement 
can actually not be responsible for the faulty behavior. This additional candidate is 
caused by the dependency for ’p2_2’. If we eliminate this dependency from statement 3, 
the dependency-based model delivers the expected results, but can no longer deal with 
aliasing. One solution would be to use two different models, one with and one without 
aliasing. The best model to be used for a specihc problem could then be chosen by the 
user (if a-priori knowledge about the problem exists) or by the use of given error class 
statistics. Similar observations hold for the DEDM and VBM. 

Structural faults. The wrong variable in the program is accessed or changed, i.e., 
the dependency graph |i8?| of the program is not structurally equivalent to the dependency 
graph of the correct program. Such faults can be repaired by replacing the variable. The 
following fragment depicts an incorrect assignment target: 

1. intx=0, y=0; 

2. X = 2; 

3. X = 2; // Should be y=2. Expected results: x=2, y=2 

For both models the debugger returns statement 1 as the only candidate but the 
expected bug cannot be located. In some cases, e.g., when line 3 reads x=3 and we 
expect the outputs x=2 and y=3, the debugger is able to locate the bug, but in general 
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more powerful solutions need to be applied, such as the introduction of replacement 
fault modes for assignments ED- 

The second subclass of structural faults is the wrong use of variables in expressions: 

1. x = 3; 

2. tmp = 2; 

3. y = 2 * tmp; // Should be y=2*x. Expected: x=3, y=6 

Because no value for tmp is specified, the debugger would return statement 2 and 
3 as diagnosis candidates regardless of model. With the VBM, we obtain the variable 
access tmp in line 3 as a single fault. Hence, this second class of structural faults can 
implicitly be handled by our models. 

7 Conclusion 

In this paper we have extended earlier work on dependency-based models of imperative 
programs by the description of dependencies for object references that allow the diagno- 
sis of situations involving aliasing, and discuss the tradeoffs inherent in the models. We 
also present results gained from experimenting with the implementation that incorpo- 
rates both, these models as well as, for the first time, the implementation of a value-based 
model of imperative programs. Both models represent an improvement over the earlier 
dependency-based model in terms of diagnosis discrimination. Runtime performance 
of the dependency-based models is satisfactory, while the value-based model still has 
to be improved in runtime performance. However the experiments have shown it to be 
superior to the dependency based models and stepwise manual debugging in terms of 
required user interaction. 
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Abstract. An effective method is presented for deriving state knowledge 
in the presence of sensing actions. It is shown how conditional plans 
can be inferred with the help of a generalized concept of plan skeletons 
as search heuristics, which allow the planner to introduce conditional 
branching points by need. 



1 Introduction 



The problem of modeling sensing actions has gained much attention in the re- 
cent past as an important step for the development of extensive foundations for 
Cognitive Robotics. Several solutions to the technical Frame Problem have been 
generalized to reasoning about the knowledge of a robot and the effect of sensing, 
e.g., in the Situation Calculus HH and the Fluent Calculus m- Based on general 
first-order logic, these approaches are sufficiently expressive to allow for mod- 
eling actions with knowledge preconditions, sensing of non-atomic properties, 
and deriving implicit knowledge. Moreover, to solve planning problems involv- 
ing knowledge goals, the notion of conditional plans has been integrated ITTESl 
since it may be necessary to plan ahead different action sequences for different 
outcomes of sensing na. 

The expressiveness of general theories for conditional planning, on the other 
hand, raises the challenge to evolve inference algorithms that efficiently deal 
with the modality of knowledge. Most existing planning methods are tailored to 
restricted classes of planning problems, e.g., 191712111 ) 1111 . In particular, none of 
these systems can solve planning problems where knowledge follows implicitly: 
A well-known example is to determine acidity of a chemical solution by sensing 
the color of a Litmus strip HS|. The only existing system with a general solution 
to the Frame Problem for knowledge is based on GOLOG However, 
this Prolog implementation is not meant for planning with sensing as it does 
not allow to search for suitable sensing actions. Rather, the user is supposed 
to provide GOLOG programs where all necessary sensing actions have been 
correctly planned. This restriction to plan verification applies to other existing 
approaches as well, such as nm. 

In this paper, we present the foundations for an effective, fully automatic rea- 
soning system capable of solving planning problems which require conditional 
plans and implicit knowledge and which may involve incomplete states, non- 
deterministic actions, and knowledge preconditions as well as knowledge goals. 
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Based on the recent solution to the Frame Problem for knowledge in the Flu- 
ent Calculus m, our main technical result is a proof that, under reasonable 
assumptions, knowledge can be identified with incomplete state specifications. 
This theorem is applied to Flux (the Flu ent Calculus Executor) — a recent logic 
programming methodology for Cognitive Robotics |22| with similar motivations 
as GOLOG ^ but where state update axioms are used to solve the inferen- 
tial Frame Problem |5| and where constraints are used for encoding incomplete 
states. 

Since conditional planning is a highly complex search problem, we also adapt 
the heuristics of nondeterministic robot programs of GOLOG HH] and develop 
a generalization which allows to search for plans in the presence of sensing. 
Conditionals occurring in the plan skeleton are evaluated at planning time only if 
the state knowledge suffices to do so; otherwise, a branching point is introduced, 
leading to a conditional plan by need. Prior to presenting the results, we give a 
brief introduction to the basic Fluent Calculus and Flux. 

2 The Fluent Calculus for Knowledge and Sensing 

2.1 State Update Axioms 

The basic Fluent Calculus combines, in pure classical logic, the Situation Cal- 
culus with a STRIPS-like solution to the representational and inferential Frame 
Problem EH- The standard sorts ACTION and SIT (i.e., situations) are inher- 
ited from the Situation Calculus m along with the standard functions Sq : SIT 
and Do : action x sit h> sit denoting, resp., the initial situation and the 
successor situation after performing an action; furthermore, the standard pred- 
icate Pass : ACTION X SIT denotes whether an action is possible in a situation. 
To this the Fluent Calculus adds the sort state with sub-sort fluent along 
with the pre-defined functions 0 : state; o : state x state i— >■ state; and 
State : SIT >->■ state; denoting, resp., the empty state, the union of two states, 
and the state of the world in a situation. Based on this signature, the Fluent 
Calculus provides a rigorously logical account of the concept of a state being 
characterized by the set of fluents that are true in the state. The following foun- 
dational axioms serve this purpose. They are a suitable subset of the Zermelo- 
Fraenkel axioms, stipulating that function o behaves like set union with 0 as 
the empty set0 

Zi O (Z 2 O Zg) = (zi O Z 2 ) O Z 3 ~^Holds{f, 0) 

Zio Z 2 = Z 20 Zi Holds{fi,f) D / = /i 

zoz = z Holds{f, zi o Z 2 ) D Holds{f,z\) V Holds{f,Z 2 ) 

z o 0 = z (V/) {Holds{f, Zi) = Holds{f, Z 2 )) D Zi = Z 2 

(V<?>)(3z)(V/)(7foZds(/,z) = <?(/)) 

^ Free variables in formulas are assumed universally quantified. Variables of sorts 
ACTION, SIT, fluent, and STATE shall be denoted by the letters a, s, /, and 
z, resp. The function o is written in infix notation. 
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where ^ is a second-order predicate variable of sort fluent and the macro 
Holds means that a fluent is contained in a state: 

Holds {f, z) = {3z') z = f o z' (1) 

The very last one of the foundational axioms above stipulates the existence of 
a state for all possible combinations of fluents. A second macro, which reduces 
to do, is used for fluents holding in situations: 

Holds{f,s) Holds{f, State{s)) 

Consider, e.g., the fluent terms OnTable{x), Acidic{x), Carries(x), and 
Red{y), denoting, resp., whether a chemical solution x is on the table, x is 
acidic, the robot carries x, and Litmus strip y is redUlThe following incomplete 
state specification says that initially there are three chemical solutions A, B, 
and C on the table, litmus paper P is not red, the robot carries nothing, and 
either i? or C is not acidic: 

Holds{OnTable{A), Sq) A Holds{OnTable{B) , S'q) A Holds{OnTable{C), Sq) 

A -'Holds {Red (P), Sq) A {Vx) ~'Holds{Carries{x), Sq) (2) 

A [-'Holds{Acidic{B) , Sq) V ~'Holds{Acidic{C), S'o)] 

Assuming uniqueness of names for all fluents, the macro definitions and the 
foundational axioms imply that m is equivalent to 

(3z) {State{So) = OnTable{A) o OnTable{B) o OnTable{C) o z 
A~'Holds{Red{P), z) A (Wx) ~'Holds{Carries{x), z) 

A [-^ Holds {Acidic{B) , z) V ~'Holds{Acidic{C), z)] (3) 

A ~'Holds{OnTable{A), z) A ~'Holds{OnTable{B), z) 

A ~'Holds{OnTable{C), z ) ) 

The reader may notice that the constraints on sub-STATE z not only reflect the 
negated Holds statements of 0 but also the fact that neither of OnTable{A), 
OnTable{B) , or OnTable{C) re-occurs. This will allow to quickly infer the result 
of removing any of these fluents from State{So) as a negative effect. 

The Frame Problem is solved in the Fluent Calculus using so-called state 
update axioms, which specify the difference between the states before and after 
an action. The axiomatic characterization of negative effects, i.e., facts that 
become false, is given by this inductive abbreviation, which generalizes STRIPS- 
style update to incomplete states: 

z' = z — f '= [z' o f = z\/ z' = z] A ~'Holds{f, z') 

z' = z - {fi o ... o fn o /„+i) = 

(3z") (z" = z - (/i O . . . O /„) A z' = z" - fn+l) 

^ This scenario, which will be used throughout the paper, is a variation of an example 
first used in |18j. 
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On this basis, the following is the general form of a state update axiom for a 
(possibly nondeterministic) action A{ x ) with a bounded number of (possibly 
conditional) effects: 

Poss{A{x), s) D (3yi) (Z\i A State{Do{A{x), s)) = (State{s) o 

V ... V 

(3y„) (Z\„ A State{Do{A{x), s)) = {State{s) o — d“) 

where the sub-formulas Ai(x , y i, State{s)) specify the conditions on State{s) 
under which A{x) has the positive and negative effects 'df and , resp. 
Both and d~ are state terms composed of fluents with variables among 
X , y i.li n = 1 and Z\i = True, then action A{ x ) does not have conditional 
effects. If n > 1 and the conditions Ai are not mutually exclusive, then the 
action is nondeterministic. 

Consider, e.g., the ACTION terms Take{x) and Test{x,y) denoting, resp., 
the robot taking x off the table and testing x by inserting Litmus paper y. 
The effects of these two actions can be defined by these state update axioms: 
Poss{Take{x), s) D 

State{Do{Take{x) , s)) = {State{s) o Carries(x)) — OnTable{x) 
Poss{Test{x,y), s) Z) 

[Holds{Acidic{x), s) A State{Do{Test{x,y), s)) = State{s) o Red{y)] V 
[-'Holds{Acidic{x), s) A State{Do{Test{x,y), s)) = State{s)] 

Put in words, taking x has the effect that the robot carries x and x is no longer 
on the table; and testing x with the help of Litmus paper y causes y to turn 
red if the solution is acidic, otherwise nothing changes. The action preconditions 
shall be defined by: 

P OSS { Take (x),s) = Holds{OnTable{x),s) 

Poss{Test{x,y),s) = True 

Recall formula (0. The state update axiom for Take{x) and the foundational 
axioms imply 

(3z) (State{Do{Take{A), Sq)) = OnTable{B) o OnTable{C) o zo Carries{A) 

A -^Holds\OnTable{A) , z ) ) 

Besides the positive effect Carries (A), the right hand side of the equation in- 
cludes all fluents which are not affected by the action. Moreover, facts given 
in m as to which fluents do not hold in z apply to the new state just as well 
as it includes z. Thus all unchanged knowledge continues to hold without the 
need to apply extra inference steps. 

2.2 FLUX 

The programming language Flux is a recent implementation of the Fluent Cal- 
culus based on Constraint Logic Programming m- Its distinguishing feature is 
to support incomplete states, which are modeled by open lists of the form 
ZO = [FI, . . . ,Fm I Z] 
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(encoding the state description ZO = FI o . . . o Fm o Z), along with constraints 
not_holds(F, Z) 

not_holds_all([Xl, . . . ,Xk] , F, Z) 

encoding, resp., the negative statements (3 y ) -i_ffoZds(F, Z) (where y are the 
variables occurring in F) and (3 y )(VX1, . . . , Xk) -iffoWs(F, Z) (where y are 
the variables occurring in F except XI, , Xk). These two constraints are used 
to bypass the problem of ‘negation-as-failure’ with incomplete states. In order 
to process these constraints, so-called declarative Constraint Handling Rules p] 
have been defined and proved correct under the foundational axioms of the Flu- 
ent Calculus. In addition, the core of Flux contains definitions for holds (F,Z), 
by which is encoded macro dO, and update (Z1 ,ThetaP,ThetaN,Z2) , which en- 
codes the state equation Z2 = (ZloThetaP)— ThetaN. The following, for instance, 
is the Flux encoding of our state update axioms (@|) (ignoring preconditions) 
and the initial specification 0: 

state_update(Zl , take(X), Z2) 

update (Zl, [carries (X) ] , [on_table(X)] , Z2) . 

state_update (Z1 , test(X,Y), Z2) 

holds(acidic(X) , Zl) , updateCZl, [red(Y)] , [] , Z2) ; 
not_holds (acidic (X) , Zl) , update(Zl, [] , [], Z2) . 

init(ZO) 

holds(on_table(a) , ZO) , 

holds (on_table (b) , ZO) , holds (on_table (c) , ZO) , 
not_holds (red(p) , ZO) , not_holds_all ( [X] , carries (X), ZO) , 
(not_holds (acidic (b) , ZO) ; not_holds (acidic (c) , ZO)), 
duplicate_free (ZO) . 

where the constraint duplicate_free(Z) means that list Z does not contain 
multiple occurrences. Suppose, e.g., that Litmus paper P is red after testing 
solution B, then it follows that B must have been acidic but not C: 

?- init(ZO), state_update (ZO , test(b,p), Zl) , holds (red (p) , Zl) . 

ZO = [on_table (a) ,on_table(b) ,on_table(c) , acidic (b) I _Z] 

Constraints : 

not_holds (acidic (a) , _Z) 



2.3 Knowledge Update Axioms 

To represent knowledge in the Fluent Calculus and to reason about sensing 
actions, the predicate KState : SIT x state has been introduced in P3|- An 
instance KState{s, z) means that according to the knowledge of the planning 
robot, z is a possible state in situation s. A fluent is then known to hold (resp. 
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not hold) in a situation just in case it is true (resp. false) in all possible states; 
and it is known whether a fluent holds just in case it is known to hold or known 
not to hold: 



Knows{f, s) = {\/z) {KState{s, z) D Holds{f, z)) 

Knows {-if, s) = (Vz) {KState{s, z) D ~'Holds{f, z)) (5) 

Kwhether{f, s) = Knows{f, s) V Knows(~>f, s) 

These macros generalize to the knowledge of arbitrary non-atomic formulas in a 
natural way. A foundational axiom stipulates correctness of state knowledge: 

KState{s, State{s)) 

The Frame Problem for knowledge is solved by axioms that determine the 
relation between the possible states before and after an action. More formally, 
the effect of an action A{x), be it sensing or not, on the knowledge is specified 
by a knowledge update axiom, 

Poss{A{x),s)z^ , . 

{yz){KState{Do{A{x),s),z) = {3z'){KState{s,z')A'I'{z,z',s))) 

In case of non-sensing actions, formula ^ defines what the robot knows of the 
effects of the action. In case of sensing actions, formula restricts the possible 
states in such a way that the sensed property becomes known. In particular, let 
the generic ACTION term Sense{f) denote sensing whether a fluent / holds, 
then: 

Poss{Sense{f), s) D 

KState(Do(Sense(f),s),z)= (7) 

KState{s, z) A [Holds{f, z) = Holds{f, s)] 

That is to say, among the states possible in s only those are still possible after 
sensing which agree with the actual state of the world as far as the sensed fluent 
is concerned. A crucial immediate consequence is that sensing always causes the 
truth value of a property to be known mB 

Based on knowledge update axioms, the inferential Frame Problem for knowl- 
edge is solved with the help of a simple inference schema. Suppose given an axiom 
which summarizes all that is known of a situation s, that is, KState{s, z) = ^{z). 
Suppose further, for the sake of argument, that Poss{A{ x),s), then (01 entails 

KState{Do{A{ a; ), s), z) = (3z') (^(^0 ^ s)) (8) 

which provides a specification of what is known in the successor situation. 

In accordance with the classical notion of planning by deduction, conditional 
plans in the Fluent Calculus are first-order citizens, composed of the primitive 

® While for the sake of abstraction axiom o specifies an ideal sensor for qualitative 
fluents, nondeterministic knowledge update axioms can be used to model sensor 
noise when sensing quantitative fluents. 
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actions of a domain and using the standard functions e (empty action), oi; 02 
(sequential composition), and //(/, 01 , 02 ) (conditional branching). Precondi- 
tions, state update, and knowledge update for these ACTION functions are de- 
fined by foundational axioms |23| . Here is an example of a situation representing 
a conditional plan: 

S = Do{If{Red{P), Take{C), Take{B)), , . 

Do{Sense{Red{P)), Do{Test{B, P), So))) 

Applied to our example initial specification, 0 , this plan can be proved to 
achieve the goal of getting a chemical solution which is known not to be acidic: 

(3x) {Knows {Carries (x), S) A Knows{~<Acidic{x), S)) 



3 Identifying Knowledge with Incomplete States 

While the explicit notion of possible states leads to an extensive framework for 
reasoning about knowledge and sensing, automated deduction becomes consid- 
erably more intricate by the introduction of the modality-like KState predicate. 
In this section, we develop the foundations for an inference method which avoids 
separate update of knowledge and states. To this end, we show how knowledge 
updates are implicitly obtained by progressing an incomplete state through state 
update axioms. 

Our approach rests on two assumptions. First, the planning robot needs to 
know the given initial specification d>{State{So)) , and this is all it knows of So, 
that is, KState{So,z)=<d>{z). Second, the robot must have accurate knowledge 
of its own actions: 

Definition 1. A set of axioms B represents accurate effect knowledge if for 
eaeh non-sensing ACTION function A, S contains a unique state update axiom 

Poss{A{ x),s)Z) Pa{z/ State{Do{A{ x ), s)), z / State {s))} (10) 

(where Pa{x,z,z') is a first-order formula with free variables among x^z,z' 
and without a sub-term of sort SIT^ and a unique knowledge update axiom which 
is equivalent to 



Poss{A{x),s) D 

iy z) {K State {Do {A{x),s),z)= (11) 

{3z'){KState{s , z') A Pa{x, z, z')) ) 

Put in words, the possible states after a non-sensing action are those which would 
be the result of actually performing the action in one of the previously possible 
states. 

Accurate knowledge of effects suffices to ensure that the possible states after 
a non-sensing action can be obtained by progressing a given state specification 
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through the state update axiom for that action. The effect of sensing, on the 
other hand, cannot be obtained in the same fashion. To see why, let 5" be a 
situation and consider the knowledge specification 

KState{S, z) = [Holds{Red{P), z) = Holds{Acidic{A)^ z)] (12) 

(which may have been inferred as the result of a Test{A,P) action). Sup- 
pose that Poss{Sense{Red{P)), S), then the knowledge update axiom (3) foi' 
Sense{Red{P)) yields two models for KState{Do{Sense{Red{P)), S), z), the 
first of which satisfies 

KState{Do{Sense{Red{P)), S), z) = 

Holds{Red{P), z) A Holds{Acidic{A) , z) 

(for all z) whereas the other one satisfies 

KState{Do{Sense{Red{P)), S), z) = 

~'Holds{Red{P), z) A ~'Holds{Acidic{A), z) 

(again for all z). The first model represents the case where Red{P) actually 
holds in S while the second model represents the case where Red{P) actu- 
ally does not hold in S. Due to the existence of these two models there can 
be no unique specification of the form KState{Do{Sense{Red{P)) , S) , z) = d>{z) 
entailed by dnj and (0. Hence, the effect of a sensing action cannot be obtained 
by straightforward progression. 

In order to account for different models for KState caused by sensing, we 
introduce the notion of a sensing history as a finite, possibly empty list of 
O’s and I’s. A history is meant to describe the outcome of each sensing action 
in a sequence of actions. For the sake of simplicity, we assume that the only 
sensing action is the generic Sense(f) with knowledge update axiom 0 and 
state update axiom State{Do(Sense{f), s)) = State(s). 

For the formal definition of progression we also need the notion of an action 
sequence cr as a finite, possibly empty list of ground ACTION terms. An action 
sequence corresponds naturally to a situation, which we denote by Sa'- 

5[] and t ) I "=' Do{A{ t),S,) 

We are now in a position to define, inductively, a progression operator P{a, g, z), 
by which an initial state specification d>{State{So)) is progressed through an 
action sequence a wrt. a sensing history g, resulting in a formula specifying z: 

P{[],g,z)^A^<l>{z) ifc=[] (13) 

P{[A{t ) I a], z) (3z0 {P{a, g, z') A T^(t, z, z')) 

if A non-sensing with state update 

P([Sense(f) | cr],<?, z) = P{a,g',z) A ~^Holds{f, S^) if c = [0 | c'] 

V{a,g' ,z) MIolds{f,Sa) if c = [1 1 <?'] 



( 15 ) 
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In case the length of the history ^ does not equal the number of sensing actions 
in (T, we define as False. As the main result, progression provides a 

provably correct inference method for knowledge update □ 

Theorem 2. Consider the initial state and knowledge Eq = {<P{State{So))., 
KState{So, z) =<P{z)} and let E be the foundational axioms plus a set of do- 
main axioms representing accurate effect knowledge. Let a be an action sequence 
such that E U Eq ^ POSS{a). Then for any model Ai of Eq\J E and any 
valuation v, 

\= KState{Sa, z) iff \= z) for some c; 

Proof (sketch) The proof is by induction on a. The base case cr = [] follows 
by (US) and Eq. The induction step for a = [A( t)\a'] with A{ t ) being a non- 
sensing action follows by oni) and knowledge update axiom (HD. The induction 
step for a = [Sense{f) \ a'] follows by (ESI) and knowledge update axiom O- 

This theorem serves as the formal justification for the Flux encoding of 
knowledge and sensing. The generic sensing action Sense{f) is encoded by a 
state update axiom which carries as additional argument the result of sensing, 
where the sensing value is either 0 or 1: 

state_update(Z, sense(F), Z, SV) 

not_holds(F, Z) , SV=0 ; holds (F, Z) , SV=1. 

The definition of progression is a direct encoding of dinj-iiii): 

p( [] , [] , Z) init(Z) . 
p([A|S] , H2, Z2) p(S, HI, Zl) , 

( state_update (Zl , A, Z2) , H2=H1 ; 

state_update(Zl, A, Z2, SV) , H2=[SV|H1] ). 

The Flux definitions for Knows{f,s), Knows{->f, s), and Kwhether{f, s) then 
follow from Theorem Q 

Corollary 3. Let (j) be a fluent term. Under the assumptions of TheoremX^ 

1. EqUE 1= Knows{4>, Scr) iff there is no model Ai of EqCE, no valuation 
V, and no history g such that Ai,v ^ 'P{a,g,z) A ->Holds{(l), z) . 

2. EqCE ^ Knows{->(f), Sr,) iff there is no model Ai of EqUE, no valuation 
V, and no history g such that Ai,v ^ 'P{a,g,z) A Holds{4>, z) . 

3. AoUA 1= K wh ether {(j), S„) iff there is no model Ai of EqUE , no valuation 
V, and no history g such that A4,v ^ V{a,g,Zi)AHolds{(f>^Zi)AV{a,g,Z 2 )A 
->Holds{4>, Z 2 ). 

Proof (sketch) Follows from Theorem 13 and macro ©• 

Hence: 

Below, POSS{a) means that a is possible in So, that is, POS'S'([])'= True and 
POSS{[A(t )\a])''A‘ POSS{a) A Poss{A{t ),Sa). 
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knows (F, S) is_f luent (F) , \+ ( p(S, Z) , not_holds(F, Z) ). 
knows(-(F), S) is_f luent (F) , \+ ( p(S, Z) , holds(F, Z) ). 

kwhether(F, S) is_f luent (F) , \+ ( p(S, H, Zl) , holds(F, Zl) , 

p(S, H, Z2), not_holds(F, Z2) ). 

where is_f luent shall be true if the argument constitutes a fluent term of 
the language. Recall, for instance, the example initial state of SectionEl Whether 
solution A is acidic is still unknown after testing it but will be known after 
further sensing the color of the Litmus strip — though it cannot be predicted 
that it is acidic (nor, of course, that it is not): 

?- \+ kwhether (acidic (a) , [test (a,p)] ) , 

kwhether (acidic (a) , [sense (red (p) ) ,test(a,p)] ) , 

\+ knows (acidic (a) , [sense(red(p)) ,test(a,p)] ) . 

yes 

The range of Theorem 0 includes nondeterministic actions. The latter may in 
particular cause loss of knowledge HZ]; e.g., 

state_update (Zl , dilute(X), Z2) 

Z2 = Zl ; update (Zl, [] , [acidic (X)], Z2) . 

?- kwhether (acidic (a) , [dilute(a) ,sense(red(p)) ,test(a,p)] ) . 
no 

4 Conditional Plans and Plan Skeletons 

The reified conditional plans of the Fluent Calculus are encoded in Flux as 
possibly nested lists of actions, in the order of execution; e.g., 

[test(b,p), sense (red(p) ) , if (red(p) , [take(c)] , [take(b)] )] 

represents conditional plan (0 from above. A planning problem with incom- 
plete states and sensing actions is the problem of finding a conditional plan 
which can be proved to be executable and to achieve the goal under any cir- 
cumstances. Therefore, if a conditional action is inserted into a plan, then each 
branch must be searched individually. To this end, we introduce the auxiliary 
actions Commit{f) and Commit which, formally, do not affect the world 
state but the knowledge: 

KState{Do{Commit{f), s), z) = KState{s, z) A Holds{f, z) 
KState\Do\Commit\-^f),s),z) = KState{s, z) A ~'Holds{f, z) 

In terms of Flux: 

state_update(Z, commit(F), Z) : - \+ F = -(_), holds(F, Z) . 
state_update(Z, commit (- (F) ) , Z) :- not_holds(F, Z) . 
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In principle, the Flux clauses we arrived at can readily be used by a simple 
forward-chaining search algorithm. Enumerating the set of plans, including all 
possible sensing actions, a solution will eventually be found if only the problem 
is solvable. However, planning with incomplete states usually involves a consid- 
erable search space, and the possibility to generate conditional plans only en- 
larges it. The concept of nondeterministic robot programs has been introduced 
in GOLOG as a powerful heuristics for planning, where only those plans are 
searched which match a given skeleton This avoids considering obviously 
useless actions such as ineffectual sensing. In the following, we generalize this 
concept to incomplete states and state knowledge. Our major extension concerns 
conditionals, which we resolve at planning time only if the state knowledge suf- 
fices to do so; otherwise, a branching point is introduced, leading to a conditional 
plan by need. 

Similar to GOLOG we use a macro do{S,a,p) where (5 is a robot program 
(represented as sequence of commands), a a sequence of actions (possibly in- 
cluding the auxiliary Commit), and p is a (possibly conditional) plan. The 
intended reading is that executing 6 in situation S'g. may result in the exe- 
cutable plan p. The crucial extension to GOLOG is this new definition of a 
conditional: 

do(if / then^i else (52, cr, p) =* (3pi,p2) {Kwhether{f, Sa) A 

do(Si,\Commit(f)\a],pi) A 

P= [If if, Pi, P 2 )] 

The other standard macros of GOLOG are straightforwardly adapted to the 
Fluent Galculus. We just mention those which will be used in our example below, 
namely, primitive actions, testing, and nondeterministic choice of sub-programs 
(denoted by ^i#(52) and of arguments (denoted by (7ra;)(5). 

do{[],a,p) = p=[] 

do{\a\5\,a,p) {3p') iPoss{a,S„) A 

do{5,[a\a\,p') A p = [o|p']) 

do {[{-,} Knows i f )7 \S], a, p) = Knowsi{-,}f,cr) A do{S,a,p) 
do {[{-,} Kwhether if )7 \ S],a,p) '= {->}Kwhetherif,a) A do{5,a,p) 

do([5i#(52 I 5], cr,p) = do{5\ + 5,(j,p) V do(^i -I- cr,p) 
doiiTTx)S,cr,p) = (3x) do(5, cr,p) 

where i5 -I- (5' denotes concatenation of two programs. The encoding in Flux is 
straightforward; we just mention the clause which encodes the conditional: 

do([if (F,E1,E2) IL] , S, P) :- 
is_f luent (F) , 

append(El, L, LI), append(E2, L, L2) , 
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( knows (F, S) , !, do (LI, S, P) ; 
knows (-(F), S) , !, do(L2, S, P) ; 
kwhether(F, S) , do(Ll, [commit (F) I S] , PI), 

do(L2, [commit (-(F)) Is] , P2) , 

P = [if (F,P1,P2)1 ). 

That is to say, if the condition can be decided in advance, then the corre- 
sponding branch is chosen; otherwise, a conditional plan is generated and both 
branches are searched. Regarding the latter case, notice that it is checked (us- 
ing kwhether) that it will be possible to evaluate the condition at execution 
time (c.f. CED) ; if not, then the clause fails as the resulting plan would not be 
executable. 

The empty robot program terminates successfully with the empty plan if the 
planning goal is satisfied: 

do([], S, []) :- goal(S). 

As an example, consider the following recursive robot program, which can 
be used to find among any selection of chemical solutions a non-acidic one with 
a sufficient supply of Litmus paper: 

proc (f ind_non_acidic , [pi(x, [(knows(on_table(x)))?, 

(not knows (acidic (x) ))? , 

[] # [test_acidity (x)l , 
if (acidic (x) , [f ind_non_acidic] , 
[take(x)] )])]). 

proc (test_acidity (X) , [(not kwhether (acidic (X)))?, 

pi(y, [test(X,y) , sense (red(y) )] )] ) . 

Put in words, to find a non-acidic solution, pick one which is not known to be 
acidic. It may be necessary to test the solution. (The first item in the body of 
the auxiliary procedure test_acidity avoids redundant testing.) If the selected 
solution is acidic, try to find another one, else grab it. 

Consider, now, the goal to get a non-acidic solution, 

goal(S) :- knows (carries (X) , S) , knows (- (acidic (X) ) , S) . 

With suitable domain clauses defining is_f luent and the action preconditions, 
the program will generate the following plan given the example initial specifica- 
tion of Section 

?- do ( [f ind_non_acidic] , [] , P) 

P = [test(b,p), sense(red(p) ) , if (red(p) , [take(c)] , [take(b)l )] 

The reader may notice that it suffices to test solution B; if it turns out to be 
acidic, then C must be non-acidic0 It is worth stressing that even with the 

® A second solution to the planning problem is of course to test solution C and to 
branch upon the result accordingly. 



378 



M. Thielscher 



given plan skeleton, it is necessary to find the right sensing action. In particular, 
the system has to backtrack over the attempt to test solution A (which renders 
unusable the only available Litmus paper)! 



5 Related Work 

A distinguishing feature of our system is its expressiveness in comparison to most 
existing systems for planning with knowledge and sensing. In 0 an implemen- 
tation is described for which a semantics is given based on the general Situation 
Calculus solution to the Frame Problem for knowledge of m- However, the im- 
plementation is based on the notion of an incomplete state as a triple of true, 
false, and unknown propositional fluents. The same representation is used in the 
logic programming systems msi, which are both given semantics by a three- 
valued variant |2| of the Action Description Language 0 . This restricted notion 
of incomplete states does not allow for handling any kind of disjunctive informa- 
tion. As a consequence, none of the aforementioned systems can solve planning 
problems that require to derive implicit knowledge (as in the Litmus scenario) 
or reasoning by cases. The latter is necessary whenever an action has conditional 
effects depending on whether some unknown fluent is true or false, but where 
both conditional effects suffice to achieve the goal j2|. Similar restrictions apply 
to the approach of [Z|, based on Description Logic. 

The only existing systems with a general solution to the Frame Problem 
for knowledge is m- However, this Prolog implementation cannot be used for 
planning with sensing as it does not allow to search for suitable sensing actions. 
Rather, the user is supposed to provide GOLOG programs where all necessary 
sensing actions have been correctly planned. Likewise restricted to plan verifi- 
cation is the approach PH, which is based on a special epistemic propositional 
logic. In contrast, our system is designed for solving planning problems as it 
allows to backtrack over sensing actions that lead to a dead end. 

The semantics of our logic program is given by previous work on integrating a 
solution to the Frame Problem for knowledge into the Fluent Galculus . This 
axiomatization technique is related to the Situation Galculus-based formalization 
of |2n|. The basic idea there is to represent state knowledge by a binary situation- 
situation relation K(s, s'), meaning that as far as the robot knows in situation s 
it could as well be in situation s' . Hence, every given fact about any such s' is 
considered possible by the robot. Having readily available the explicit notion of 
a state in the Fluent Galculus, our formalization avoids this indirect encoding 
of state knowledge, which is intuitively less appealing because it seems that a 
robot should always know exactly which situation it is in — after all, situations 
in the Situation Galculus are merely sequences of actions that have been or will 
be taken by the robot m- In view of the computational challenge raised by 
the Frame Problem for knowledge, a crucial advantage of our approach is also 
the simple inference scheme (jBD provided by the concept of knowledge update 



axioms. 
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A conceptually different semantical approach has been proposed in as an 
extension of the Action Description Language which is more powerful than the 
abovementioned |2| • Incomplete knowledge is formalized by a so-called epistemic 
state, which is a set of possible sets of possible states. Intuitively, an epistemic 
state corresponds to the set of models for our KState predicate. We therefore 
suspect that our logic program can be shown to provide a sound and complete 
proof procedure for this semantics, too, but the formal details have yet to be 
worked out. 

6 Discussion 

We have developed the formal foundations for an effective inference method for 
state knowledge in the presence of incomplete states, nondeterministic actions, 
and sensing. Conditional plans are computed by reasoning about knowledge 
based on progression and with the help of a generalized concept of nondetermin- 
istic robot programs as search heuristics. The resulting extension of the high-level 
programming language Flux exhibits a clear distinction between nondeterminis- 
tic actions and nondeterminism in the heuristics. The latter needs to be resolved 
at planning time, possibly by introducing a branching point into a plan. Nonde- 
terminism in state update axioms, on the other hand, is respected when verifying 
knowledge preconditions or proving that the plan is correct under any outcome. 

We have successfully applied Flux to the high-level control of a simple Lego 
robot P2| as well as a Pioneer-2, both of which perform delivery tasks and need 
to generate conditional plans which include sensing whether doors are closed. 

Future work will be to extend the progression operator to actions with ram- 
ifications and to concurrency, in order to provide the formal justification for 
inferring knowledge in more complex domains. Furthermore, off-line planning in 
Flux should be interleaved with on-line execution of sensing actions, following 
the argument of 0 that pure off-line planning can often be inefficient in the 
presence of sensing. Finally, while sensor noise has been ignored in all our appli- 
cations thus far, the concept of knowledge update axioms can be readily applied 
to model nondeterministic outcomes, and hence to account for noise Q. We cur- 
rently pursue the axiomatization and implementation of sensor noise within our 
approach along this line. 
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Abstract. Intelligent agent systems have been the subject of intensive research 
over the past few years; they comprise one of the most promising computing 
approaches ever, able to address issues that require abstract modelling and 
higher level reasoning. Virtual environments, on the other hand, offer the ideal 
means to produce simulations of the real world for purposes of entertainment, 
education, and others. The merging of these two fields seems to have a lot to 
offer to both research and applications, if progress is made on a co-ordinated 
manner and towards standardization. This paper is a presentation of VITAL, an 
intelligent multi-agent system able to support general-purpose intelligent virtual 
environment applications. 



1 Introduction 

Probably one of the most exciting and promising scientific fields ever, the field of 
intelligent agents is still the subject of major controversy over its origin, formal 
background, definitions, methods, applications and future directions. The notion of an 
intelligent agent, indisputably challenging to define precisely, has been used to 
characterize a vast number of approaches and applications, ranging from simple 
softbots to complex, large-scale industrial control systems. 

Recent attempts to merge intelligent agent approaches with virtual reality and 
artificial life have given birth to the field of intelligent virtual environments (IVEs). 
An IVE is a virtual environment resembling the real world (or similar), inhabited by 
autonomous intelligent entities exhibiting a variety of behaviours. These entities may 
be simple static or dynamic objects (a revolving sun, traffic lights, etc.), virtual 
representations of life forms (virtual animals and humans), avatars of real-world users 
entering the system, and others. In fact, the structure and contents of a virtual 
environment are only restricted by the nature of the target application and the 
designer’s imagination - and, of course, the amount of computing power available. 

Today, IVEs are employed in a variety of areas, mainly relating to simulation, 
entertainment, and education. Sophisticated simulated environments of different types 
(open urban spaces, building interiors, streets, etc) can significantly aid in 
architectural design, civil engineering, traffic and crowd control, and others. In 
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addition, precisely modelled simulations of real-world equipment (vehicles, aircrafts, 
etc) not only can be tested at reduced cost and risk, but also more accurate results can 
be obtained thanks to the additional element of control by and interaction with 
intelligent, thus closer to real life, entities. Moreover, IVEs have set new standards in 
computer-aided entertainment, through outstanding examples of computer games 
involving large, life-like virtual worlds (where imaginative scenarios are to be 
challenged), interactive drama (where the user is an active participant in the plot) 
virtual story-telling, and many other areas where immersion and believability are key 
factors. Concluding, IVE-based educational systems incorporate believable tutoring 
characters and sophisticated data representation techniques, resulting in the 
stimulation of user interest and perceptual ability, thus providing a novel, effective 
and enjoyable learning experience. 

Despite the fact that an intelligent agent is the ideal metaphor for representing 
intelligent inhabitants inside an IVE, surprisingly little effort has been directed 
towards a formal and co-ordinated merging of intelligent agent systems and virtual 
reality techniques to produce IVEs fully exploiting the advantages of both fields. This 
paper is a presentation of our attempt to contribute to such an initiative: a fully 
functional intelligent agent system with the ability to support virtual intelligent agents 
embodied inside simulated worlds represented using VR techniques. Along with a 
number of other features of both practical and scientific value, the system can be used 
for a variety of purposes, acting as either an IVE for one of the application areas 
mentioned above, or a typical multi-agent system employed in classical application 
domains, such as control systems, distributed problem solving, resource allocation 
and many others. In the rest of this paper, a discussion of relevant research work is 
given in section two. Section three is a thorough presentation of the proposed system, 
while section four is a layout of additional work carried out towards multi-agent 
support. An example of the system’s operation is given in section five. 



2 Related Work 

The Beliefs-Desires-Intentions (BDI) model [4] is probably the most popular 
approach towards the design of intelligent agents, mainly due to its ability to trigger 
behaviours driven by conceptually modelled intentions and goals rather than explicit 
procedural information. In addition, it seems to be a functional abstraction for the 
higher-level reasoning processes of the human mind, those that are related to action 
selection and the focusing of intelligent reasoning processes on specific desired states. 
The BDI model has been adopted in a significant number of implementations: 

In [3], Bratman et al. present the Intelligent Resource-bounded Machine 
Architecture (IRMA), an architecture for resource-bounded (mainly in terms of 
computational power) deliberative agents, based on the BDI model. IRMA agents 
consist of four main modules: a means-end planner, an opportunity analyser, a 
filtering process and a deliberation procedure. In addition, they contain a plan library, 
and data structures to store beliefs, desires and intentions. 

Jennings in [10] proposes GRATE, an architecture clearly focused on co-operative 
problem solving through agent collaboration. Central to the entire architecture is the 
notion of joint-intentions. In fact, even though GRATE is a deliberative architecture 
based on the BDI model, it is specifically referred to as a belief-desire-joint-intention 
architecture. 
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The BDI model has provided valuable theoretical grounds upon which the 
development of several other architectures and approaches, such as hybrid and 
layered agents, was based [9]: 

The Procedural Reasoning System (PRS) [7] is a hybrid system, where beliefs are 
expressed in first-order predicate logic and desires represent system behaviours 
instead of fixed goals. PRS includes a plan library containing a set of partial plans, 
called knowledge areas, each associated with an invocation condition. Knowledge 
areas might be executed due to goal-driven reasoning or as a response to sensory data; 
this way, the agent is capable for both deliberative and reactive behaviours. 

Muller in [12] proposes INTERRAP, a layered agent architecture focusing on the 
requirements of situated and goal-directed behaviour, efficiency and co-ordination. 
INTERRAP agents consist of a world interface, a behaviour-based, a plan-based and a 
co-operation component, each affecting agent behaviours at a different level of social 
and functional abstraction.. 

In [16], Sycara et al. present the Reusable Task Structure-based Intelligent 
Network Agents (RETSINA) architecture. The architecture consists of three types of 
agents: interface, task agents and, information agents. 

Due to its apparent focus on high-level reasoning and generation of elaborate 
behavioural patterns, the BDI model seems to be inadequate to efficiently and 
effectively model all aspects of intelligent reasoning. However, any system that needs 
to exhibit goal-driven behaviour should incorporate, among others, a BDI-based or 
equivalent component. 

The merging of intelligent agent systems, artificial life and classical VR techniques 
has given birth to the field of Intelligent Virtual Environments (IVEs). Typical 
examples involving IVEs and general virtual agents include Humanoid [2], Creatures 
[8], Artificial Eishes [17], and others. 

The CoMMA-COGs [5] project (Cooperative Man Machine Architectures - 
Cognitive Architecture for Social Agents) is an architecture for Multi-Agent systems 
and animated virtual environments, developed by the German Research Center for 
Artificial Intelligence. The system employs traditional multi-agent research 
approaches. Eurthermore, it supports self-organization of agent societies, so that 
external users perceive them as units, and, thus, being unaware of the underlying 
organization processes. In addition, resource-awareness allows agents to perform In 
unpredictable environments while flexibly managing their resources. In general, IVEs 
tend to focus on either the virtual representation and embodiment side, or the 
intelligence side. Eull benefit has not yet been taken of the combined advantages of 
intelligent multi-agent systems and virtual environments. A complex, accurately 
modelled and general-purpose IVE, inhabited by numerous believable entities driven 
by strong and effective AI reasoning processes, is yet to be presented. 

A predecessor to the VITAL system and a first effort towards an intelligent agent 
system architecture with the ability to support IVE applications, the DIVA 
architecture, developed by the Knowledge Engineering Lab of the University of 
Piraeus, was presented in [18]. 



3 Overview of the VITAL System 

The system presented in this paper is called VITAL, an acronym for Virtual 
InTelligent Agents with Logic. It represents an attempt to explore the world of 
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intelligent agent systems and intelligent virtual environments, initiated in UMIST, UK 
[1]. VITAL is a simulation of a maze world. Agents are required to explore a given 
maze, locate specific items inside it and process them in response to user instructions, 
e.g. move them to given locations. Agents have no initial knowledge of either the 
structure of the maze or of the items’ locations. The system is monitored using a 3D 
viewer supporting free world navigation and several configuration options. 

The system was designed and implemented with the following goals and 
requirements in mind; 

1. Wide range of applications: the system should support a wide range of areas. 

2. Agent effectiveness, independence and agility: for a given area, agents should be 
effective, that is, achieving what they have been assigned; furthermore, they should 
require no user intervention while doing so; finally, they should be agile, 
maintaining their effectiveness even when the environment changes unpredictably 
within the limits of a defined domain. 

3. Distributed and modular structure: the system should be distributed, consisting of 
discrete co-operating components executing on possibly different machines across 
a network; in addition, modularity should allow the insertion and removal of 
components at runtime without interrupting the system’s operation. 

4. Significant research potential: the system should be designed so that scientific 
experimentation is inherently. 

5. Intuitive observation and monitoring capabilities. 

6. Sophisticated implementation: the system should be implemented using modern, 
specialized software tools and latest technologies, essentially comprising a high- 
quality software product. 

7. Extendibility and reusability: the system should be highly extendible so that it can 
be advanced along with ongoing research, by allowing the introduction of new 
concepts and approaches, such as multi-agent characteristics. 



3.1 Architecture 

The VITAL system consists of three types of conceptually discrete components: 
worlds, agents and viewers. These are implemented as separate software applications 
according to the client-server approach. A world component represents a virtual 
environment inside which the entire agent system’s activity takes place. Agent 
components represent actors inside an environment. Agents perceive the environment 
and act upon it according to goal-driven behaviours. Viewers offer the means to 
human supervisors to observe the environment and all activity inside it in a domain- 
specific manner. The system’s component-based nature is outlined in Figure 1 below: 
During system operation, a number of interactions take place between components. 
In particular, when an agent wants to sense its environment, it requests sensory 
information from the world component - a sense request. The world component then 
replies, providing the requested information. Similarly, when an agent wants to 
perform an action, the corresponding agent component provides all necessary action 
information to the world component - an action request - and then the world 
component responds regarding whether the requested action was successfully applied. 
In the case of successful action application, the world component sends world change 
data to all viewer components so that actions are correctly visualised. In addition. 
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when a viewer needs to build an entirely new visualisation, it requests a full 
description of the world model - a world description request. 




WORLD 



(future extension) 




Fig. 1. VITAL system architecture outline 

The separation of the world and the agent into two discrete components essentially 
enables the separation of the logical layer from the physical, real-world layer. This 
way, agent design can remain focused on the abstract properties and characteristics of 
the domain addressed and the conceptual specifics of the problem, hence, 
significantly increasing the system’s modelling capacity. On the other hand, the world 
component can deal with all the realization details, for example, the control of 
equipment or hardware, giving physical substance to agents’ virtual actions. 

Adopting the client-server approach directly serves the goal of extendibility, since 
there is no limitation as to how many agent clients can be connected at any time to a 
world server. In addition, multiple simultaneous viewer client connections are 
supported, allowing the system to be observed and monitored from different views, 
and possibly in different ways, depending on the visualisation capabilities of each 
viewer client. 



3.2 World Modelling 

Each component maintains a complete or partial internal representation of the world 
and, in the case of an agent component, additional properties about itself. The 
representation used can be either symbolic or object-oriented, according to the 
component’s nature. An additional type of representation, called pseudo-symbolic, is 
used to transfer symbolic facts between applications as well as to translate between 
symbolic and object-oriented representations. 

In the VITAL system, worlds are modelled from the object-oriented point of view 
as sets of interconnected locations. Each location contains one or more items. Each 
item has a name, belongs to an item class and has properties. Agents are also 
represented as world items. To enrich the modelling scheme with spatial features, 
each location includes a two- or three-dimensional co-ordinate pair, according to the 
applications’ needs. 

This modelling abstraction is adequate to describe a substantial number of different 
environments: simple mazes, real-world buildings, streets, networks, etc. A world 
modelled from the object-oriented approach is essentially represented as a partially 
connected non-directed graph. 
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To implement this representation as well as to equip it with the necessary 
management functionality, a specialised class-based hierarchy was defined. 
According to it, each item has a property of name ‘class’, which denotes its item 
class, that is, the conceptually wider class of entities the item belongs to. Item classes 
can be defined according to the application’s modelling requirements to any level of 
abstraction desired. Their selection depends on the properties selected to adequately 
discriminate types of items in a world. For instance, an agent could be denoted by an 
item property of name ‘class’ and value ‘agent’, whereas a non-agent item could have 
a ‘class’ property of value ‘object’. 

The symbolic modelling methodology defined by the architecture borrows 
syntactical and semantics elements from predicate logic and logic programming. The 
VITAL system uses the symbols shown in Table 1 below, with their respective 
interpretations: 



Table 1. World modelling symbols used in the VITAL system 



Symbol 


Interpretation 


connects(Xl, Y1,X2, Y2) 


‘(XI, Yl)’ and ‘(X2, Y2)’ are connected 


at(Item, X, Y) 


The location of ‘Item’ is ‘(X, Y)’ 


location(X, Y) 


‘(X, Y)’ is a valid maze location 


item(Item) 


‘Item’ is a valid maze item 


class(Item, Class) 


The class of ‘Item’ is ‘Class’ 


<property_name>(Item, V alue) 


‘Item’ has a property with a name of 
‘<property_name>’ and a value of ‘Value’ 



To handle asynchronous network transmission of symbolic information, the 
architecture introduces a pseudo-symbolic representation, according to which, facts 
and functors are broken down to a series of strings, the first of which being the fact or 
functor name and the rest arguments. Non-atomic terms, i.e. variables, are represented 
by an appropriate keyword selected by the agent system designer, for instance, 
‘#VAR’. A terminating dot is appended after the last argument to denote that no more 
arguments should be expected. If a series of facts or functors is to be transmitted, an 
additional terminating dot is appended after the last fact or functor; thus, two 
terminating dots should be expected at the end of a series of facts or functors in 
pseudo-symbolic representation. 



3.3 World Server 

The world server can be viewed as the central component of the architecture; it 
provides the grounds on which all action takes place. In addition, it ensures that this 
action follows specific rules, maintaining consistency throughout the system at all 
times. Furthermore, from a functional point of view, the world server co-ordinates 
data interchange between applications ensuring that the agent’s model for the world 
and the viewer’s visual representation are valid. The world server is divided into three 
layers, as shown in Figure 2 below: 
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Fig. 2. World server structure 



The world representation layer contains all data and functionality required to 
represent and manage the world. The connection manager layer is an encapsulation of 
the server’s communications layer, so that the connection management programming 
interface specifically manages agent and viewer client connections as well as sensory 
and action requests - a useful programming abstraction offered to agent system 
developers. Finally, the control layer uses the functionality of the world 
representation layer to receive data from clients, carry out requests and transmit 
results. The world representation approach used in the world server is object-oriented. 

The control layer is responsible for coordinating multiple connections (an agent 
and one or more viewers). In addition, all data receipt and transmission must be 
appropriately co-ordinated so that visualisations by viewer clients are consistent with 
the world model at all times. To achieve that, the control layer operates on a state- 
dependent basis. 



3.4 Agent Client 

The agent client is the component with the most vital contribution to the architecture. 

It stands as an implementation of an actor inside an environment simulated by a world 
server; it is the ingredient that brings the system to life. It essentially introduces the 
element of intelligence thanks to innate support for intelligent problem solving. As 
shown in Figure 3, an agent client consists of an intelligence layer and an interface 
layer. The intelligence layer is further divided into the knowledge base, the decision 
engine and the set of sensors and effectors. 

Agent clients operate on a sense-decide-act cycle. During the sense stage, a 
‘SENSE’ keyword is sent to the world server; results received in pseudo-symbolic 
representation are then processed by sensors and appended to the knowledge base. 
During the decision-making stage, the decision engine reasons upon the contents of 
the knowledge base and creates a plan for one of the agent’s goals. Finally, during the 
action stage, the plan’s next action is sent to the world server in pseudo-symbolic 
representation and effectors apply effects concerning the agent’s own internal state. 

The knowledge base is the agent’s memory. It contains all data that has been 
perceived by the agent or produced as a result of its reasoning processes. Data in the 
knowledge base are formulated in a symbolic fashion, thus following the symbolic 
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Fig. 3. Agent client structure 



modelling approach. Consequently, the knowledge base readily supports the 
agent’ sintelligent reasoning processes. Furthermore, to support intentional reasoning, 
the knowledge base is structured according to the BDI (Beliefs-Desires-Intentions) 
architecture [4]. An example knowledge base is shown below: 

[ 

beliefs ( [ 
holding ( [] ) 

] ) , 

desires ( [ 
explore (X, Y) , 
pickup (theRedBall) , 
drop (theRedBall , 1, 2), 
pickup (theBlueBall) , 
drop (theBlueBall , 3, 2), 
move ( 2 , 2 ) 

] ) , 

intentions ( [ 



Abilities represent the ways an agent can act upon a world. In a VITAL agent, 
abilities are also defined within the agent’s beliefs. In the presented architecture, 
abilities are defined as {N, P, E), where N stands for the ability name, a functor that 
identifies it and provides access to all necessary arguments, P for the list of 
preconditions and E for the list of effects. Preconditions are functors that must be 
included in an agent’s knowledge base for an ability to be usable. Effects are functors 
processed by an agent’s effectors to update its beliefs. An example of an ability 
defined according to the scheme described above follows: 

ability: move(X, Y) 

preconditions: [at (CX, CY) , connects (CX, CY, X, Y) ] 
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delete: [at (CX, CY) ] 

add : [at (X, Y) ] 

The above example shows the definition for the ‘move’ ability, that is, the agent’s 
ability to move from one location to another. The preconditions list denotes that the 
ability is usable only when there is a connection between the agent’s current location 
with co-ordinates (CX, CY) and the new location with coordinates (X, Y). Effects are 
divided into the delete and add lists; these denote the facts that must respectively be 
removed from and added to the agent’s beliefs to reflect the new state produced. 

The decision engine is responsible for controlling the agent’s behaviour; it is an 
encapsulation of its very intelligence. On every decision-making stage, the engine 
validates the current plan, and if this fails, or if there is no current plan, the engine 
attempts to generate a new plan for the goal of top priority. If no plan can be 
generated for the top goal, the engine attempts to generate a plan for the next one, and 
keeps doing so until a plan is generated or until no more goals are available. 

In VITAL agents, the basic component of the decision engine is the plan generator, 
or planner. The planner consists of two layers: a) a general-purpose means-end 
planner, and b) a problem-specific second level, which employs heuristics to perform 
action selection and hence take full advantage of available knowledge into a specific 
application area, something that significantly reduces computational load and agent 
response times. 

Sensors and effectors encapsulate the agent’s sensing and acting functionality; 
essentially, they are the only access points to its knowledge base. At the end of every 
sense stage sensors process sensory data received by the world and finally append 
them to the agent’s beliefs. Moreover, after transmission of an action request, 
effectors update its knowledge base as to reflect its new internal state. 

It is important to note that the agent does not have unlimited sensing abilities. 
What is available to its sensors on any given location is decided upon by the world, 
following certain rules for vicinity, obstruction, etc. This allows agents to be 
employed in situations where environment knowledge is obtained gradually. It also 
gives a certain amount of realism to experimental simulations. The rules are 
appropriately designed to suit the needs of each application. 

Furthermore, an agent’s model of the world might not only be a subset of, but 
inconsistent with the ‘real’ one, that is, the one maintained by the world server. This 
may be a desired effect, or the result of erratic operation, most probably due to limited 
or faulty sensing and reasoning. 



3.5 Viewer Client 

The viewer client is the encapsulation of the system’s visualisation functionality. 
Similarly to the world server, it maintains an object-oriented world model, and 
contains a communications layer with sufficient translation functionality. The viewer 
client’s structure is shown in Figure 4 below: 

The viewer displays the maze and its contents as a 3D scene, through which a user 
can navigate freely to obtain a suitable observation angle. Agents are displayed using 
avatars loaded from VRML files. A number of additional features are also supported. 




390 



G. Anastassakis, T. Ritchings, and T. Panayiotopoulos 



Viewer client 



World desci^ tion requests 
Change notifications 



Fig. 4. Viewer client structure 



Control 


World representation 




Communications layer 



such as lighting, avatar file configuration, as well as locking of the viewpoint on an 
avatar from various viewing angles, so that agent movement and actions can be easily 
tracked. The viewer client’s user interface as well as the VITAL system in full 
operation is shown in Figure 5 below. 



3.6 Implementation 

Reasoning processes within the VITAL agent client have been implemented using 
SICStus Prolog [15], an implementation of the Prolog language developed at the 
Swedish Institute of Computer Science (SICS). Apart from being a fully functional 
Prolog system, offering multiple useful features such as constrained solving, access to 
operating system resources, parallel solving and many others, SICStus Prolog 
supports compilation of the Prolog code. Compiled predicates will run faster, using 
memory more economically. Compiled predicates can be called from within source 
code in another programming language, such as C++, thanks to an interface provided 
by the SICStus system. This is an essential feature if reasoning mechanisms built in 
Prolog are to serve as parts of another Win32 application. 

The planner developed for the purposes of the VITAL system is a breadth-first, 
state-reducing, means-end planner, called StateExplorer. Due to its breadth-first 
nature, it systematically explores a given state space by applying all available actions 
to produce next states for a list of current ones, hence, the StateExplorer name. The 
planner was written entirely in the Prolog language; it is defined as a plan/4 predicate. 

The VITAL agent client’s second-level planner is implemented as a plan2/3 Prolog 
predicate. It is responsible for exploiting domain- dependent knowledge into the nature 
of the agent’s abilities, to minimize computational effort. 

The visualisation engine of the VITAL viewer has been implemented using 
OpenGL. Avatars are loaded from VRML [19] files thanks to a parser implemented 
using Lex and YACC tools for the Win32 environment, namely the Parser Generator 
package by Bumble-Bee Software. The parser is capable of processing VRML files 
containing a subset of the language sufficient to represent scenes and objects exported 
from major 3D design packages, such as Kinetix 3D Studio and Macromedia Poser. 
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Fig. 5. The VITAL system in full operation 



4 Multiple Agent Support and Inter-agent Communication 

The VITAL system has recently been extended to support co-existence and 
simultaneous operation of multiple agents. The new system is named ‘mVITAL’ 
(multi-agent VITAL). It enables the development of simulations where intelligent 
agents communicate through simple speech acts, co-operate and help each other to 
achieve goals, reason on other agents, exchange beliefs, etc. 

As discussed in [1], the agent client is probably the most significant component of 
the VITAL system, since it introduces the feature of intelligence, thus allowing the 
properties of agility and dynamic behaviour to emerge. Agent clients in the extended 
mVITAL system are of even more crucial importance, since they enable the definition 
of agent societies, introducing the elements of inter-agent interaction, inter-agent 
communication, co-ordination, and distributed problem solving. 

In mVITAL, an agent can identify other agents by sensing their ‘class’ property 
and checking if it has a value of ‘agent’. In addition, they are able to refer to each 
other using the value of their ‘name’ property, just as they refer to themselves using 
the ‘me’ keyword. For instance, an agent would believe that ‘at(smith, 1, 1)’ about 
agent Smith’s position, and ‘at(me, 10, 10)’ about its own. Social reasoning is from 
that point on a matter of modelling, and agent interactions emerge as a result of 
properly defined abilities. 

Agents are able to communicate using virtual, non-visual speech items. Speech 
items belong to a special class denoted by a ‘speech’ value to the item’s ‘class’ 
property. Speech items have a ‘text’ property, whose value can be anything an agent 
wishes to communicate to another, usually text in some natural language. A speech 
act between two agents can then be modelled as the consecutive application of two 
actions, one adding a speech item to the world through the effects part, and the other 
enabling the perception of the item through the preconditions part, thus allowing the 
receiving agent to respond. Communication can be made even more complex, with 
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additional speech item properties denoting implied or ‘hidden’ information, as well as 
information related to mood, tone, and other expression-related parameters. KQML 
[6] performatives can he modelled using a suitable set of properties; such a set could 
include properties such as ‘MSG-TYPE’, ‘VERB’, etc, to denote intent, tone, and 
other information exchanged according to KQML format. To allow more intuitive 
observation, an extended viewer client can appropriately handle speech-objects and 
use property values to display communicated text on-screen, reproduce speech using a 
voice generation engine, etc. 

A crucial issue with speech items is that they need to be automatically removed 
from the world after a certain period of time, to simulate the real world analogy, 
where a person’s speech is heard only while it is spoken. In mVlTAL, speech items 
last a little longer (five seconds) so that all agents in a given range can sense them 
during a following sense-decide-act cycle. Eventually, five seconds after insertion to 
the world, speech items are automatically removed by the world server. 




Fig. 6. The client agent requests a copy of “Art of Prolog” from the librarian agent 



5 A Working Example 

A simple simulation has been modelled in the mVITAL system to demonstrate agent 
communication and information exchange. The example could originate from the 
field of computer games or simulation. 

The presented scenario involves two agents, a librarian and a library client. The 
client is looking for a book. In order to get it, it approaches the librarian and asks for 
it. Then, the librarian sets off and starts looking for the requested book among the 
library’s shelves. If the book is available, the librarian brings it to the client. 

According to the VITAL architecture, the above sequence can be modelled as 
follows: Initially, the client-type agent is committed to requesting a certain book from 
the librarian. This is denoted by a specific goal in its knowledge base, e.g. 
‘request_book(“some_title”)’. Among the effects of this goal, there is the creation of a 
speech object with a text value of ‘I want a copy of “some_title”’, or something 
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similar. After the speech object’s creation, the librarian-type agent is able to sense it, 
and extract book title information from the sensed speech object’s text value. Then, a 
goal is adopted, forcing the librarian to locate the requested book, that is, move close 
to it. The scenario then follows similarly with the adoption and execution of 
appropriate goals, forcing the librarian to return with the requested book and pass it to 
the client, or inform the client that the book is unavailable by creating an appropriate 
speech object. 




Fig. 7. The librarian is searching for the requested book while the client is waiting 



6 Conclusions and Future Work 

This paper has been a presentation of the VITAL intelligent agent system, a 
framework for developing a variety of applications ranging from typical agent-based 
control systems to VR-based simulations, to IVEs. Despite the fact that numerous 
issues still need to be addressed for the system to be used in complex evaluation 
environments such as the RoboCup-Rescue simulator [11], the system is fully 
functional, and all design requirements have already been fulfilled. In particular, the 
system is distributed, thus able to exploit the benefits of today’s sophisticated 
networking technologies and the Internet; it employs formal AI techniques - logic 
programming, planning, intentional reasoning - to support intelligent agent 
behaviours; it is modular and component-based, enabling the deployment of persistent 
applications; different types of agents - not necessarily built according to the structure 
proposed by the architecture, but using the same communication scheme - can be 
connected to a world server, providing openness and extendibility, as well as enabling 
dynamic alteration of a simulation’s structure and experimentation with other 
reasoning approaches; the system incorporates sophisticated VR techniques to 
produce intuitive and believable visualisations; finally, the system comprises a set of 
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state-of-the-art software applications, built using latest and specialized software 
engineering technologies. 

The VITAL system contributes to original AI and IVE research not by actually 
extending current methods and approaches, but by providing a robust and well- 
designed architecture serving as the grounds on which research work and extension 
can take place. This way, even though VITAL is still at an experimental stage, it eases 
work towards improvement of the underlying formal approaches, standing as a 
research testbed where changes are applied in a straightforward manner and results 
are instantly and intuitively observed; this is the system’s most important and original 
contribution to today’s research. 

As mentioned in 4, the mVlTAL system is still under construction and 
experimentation. However, evaluation of the VITAL system’s performance as a 
single agent starting point has justified the continuation of the effort towards a multi- 
agent level, showing that there is strong potential in using MAS technology in the 
field ofIVEs. 

Euture work directions include, but are not restricted to, making the system more 
open and configurable, with the ultimate goal being a powerful IVE-authoring tool 
that will effectively contribute to formalization and standardization in the field. To 
achieve that, we are already working on the merging of the system with VAL (Virtual 
Agent Language) [13], to enable abstract and dynamic definitions of agent 
personalities. VAL is a C/C-i-H-like agent-oriented programming language based on 
logic programming that was initially developed for the system presented in [14]. 
Moreover, we are addressing the issue of a world-modelling scheme general enough 
to enable precise definitions of a variety of scenarios and simulations. Eurthermore, 
we are enriching the agents’ planning abilities by introducing explicit temporal and 
spatial references. In addition, we are investigating more sophisticated visualization 
and embodiment techniques, as well as the issue of user intervention to simulations 
through embodiment and presence in the system. Einally, from an implementation 
point of view, various networking technologies are evaluated in an effort to optimise 
the system’s performance so that real-time execution is guaranteed. 
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Abstract. Ontologies will play a pivotal role in the “Semantic Web”, 
where they will provide a source of precisely defined terms that can 
be communicated across people and applications. OilEd, is an ontology 
editor that has an easy to use frame interface, yet at the same time allows 
users to exploit the full power of an expressive web ontology language 
(OIL). OilEd uses reasoning to support ontology design, facilitating the 
development of ontologies that are both more detailed and more accurate. 



1 Introduction 

Ontologies have become an increasingly important research topic. This is a result 
both of their usefulness in a range of application domains PEEIj and of the 
pivotal role that they are set to play in the development of the Semantic Web 

The Semantic Web vision, as articulated by Tim Berners-Lee ^ , is of a Web 
in which resources are accessible not only to humans, but also to automated 
processes, e.g., automated “agents” roaming the web performing useful tasks 
such as improved search (in terms of precision) and resource discovery, informa- 
tion brokering and information filtering. The automation of tasks depends on 
elevating the status of the web from machine-readable to something we might 
call machine-understandable. The key idea is to have data on the web defined 
and linked in such a way that its meaning is explicitly interpretable by software 
processes rather than just being implicitly interpretable by humans. 

To realise this vision, it will be necessary to annotate web resources with 
metadata (i.e., data describing their content/functionality). Standardisation pro- 
posals for annotation languages have already been submitted to the World Wide 
Web Consortium (W3C), in particular RDF (Resource Description Framework) 
and RDF Schema (see ^ for a discussion of the roles of these languages and of 
XML/XML Schema). However, such annotations will be of limited value to auto- 
mated processes unless they share a common understanding as to their meaning. 
Ontologies, can help to meet this requirement by providing a “representation of 
a shared conceptualisation of a particular domain” that can be communicated 
across people and applications |0|. 



F. Baader, G. Brewka, and T. Eiter (Eds.): KI 2001, LNAI 2174, pp. 396-^^^ 2001. 
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RDF Schema (RDFS) itself is already recognisable as an ontology/knowledge 
representation language: it talks about classes and properties (binary relations), 
range and domain constraints (on properties), and subclass and subproperty 
(subsumption) relations. However, RDFS is a relatively primitive language (the 
above is an almost complete description of its functionality) , and more expressive 
power would clearly be necessary/desirable in order to describe resources in 
sufficient detail. Moreover, such descriptions should be amenable to automated 
reasoning if they are to be used effectively by automated processes. 

These considerations have led to the development of OIL [Z], an ontology 
language that extends RDFS with a much richer set of modelling primitives. A 
similar RDFS based web ontology language called DAML was been developed 
as part of the DARPA DAML project 0 and the two languages have now been 
merged under the name DAML-|-OIlQ. OIL has a frame-like syntax, which fa- 
cilitates tool building, yet can be mapped onto an expressive description logic 
(DL), which facilitates the provision of reasoning services. OilEd is an ontology 
editing tool for OIL (and DAML-bOIL) that exploits both these features in order 
to provide a familiar and intuitive style of user interface with the added benefit 
of reasoning support. Its main novelty lies in the extension of the frame editor 
paradigm to deal with a very expressive language, and the use of a highly opti- 
mised DL reasoning engine to provide sound and complete yet still empirically 
tractable reasoning services. 

Reasoning with terms from deployed ontologies will be important for the 
Semantic Web, but reasoning support is also extremely valuable at the ontol- 
ogy design phase, where it can be used to detect logically inconsistent classes 
and to discover implicit subclass relations. This encourages a more descriptive 
approach to ontology design, with the reasoner being used to infer part of the 
subsumption lattice (see the case study presented in Section 0) ; the resulting 
ontologies contain fewer errors, yet provide more detailed descriptions that can 
be exploited by automated processes in the Semantic Web. Finally, reasoning is 
of particular benefit when ontologies are large and/or multiply authored, and 
also facilitates ontology sharing, merging and integration 0; considerations that 
will be particularly important in the distributed web environment. 



2 Oil and DAML+OIL 

The development of OIL resulted from efforts to combine the best features of 
frame and DL based knowledge representation systems, while at the same time 
maximising compatibility with emerging web standards. The intention was to 
design a language that was intuitive to human users, and yet provided adequate 
expressive power for realistic applications (many early DLs failed on this second 
count — see fll|). 

The resulting language combines a familiar frame like syntax (derived in 
part from the OKBC-lite knowledge model El), with the power and flexibility 

see http : // www . daml . org 
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of a DL (i.e., boolean connectives, unlimited nesting of class elements, transitive 
and inverse slots, general axioms, etc.). The language is defined as an extension 
of RDFS, thereby making OIL ontologies (partially) accessible to any “RDFS- 
aware” application. 

The frame syntax is less daunting to ontologists/domain experts than a DL 
style syntax, and it facilitates a modelling style in which ontologies start out sim- 
ple (in terms of their descriptive content) and are gradually extended, both as 
the design itself is refined and as users become more familiar with the language’s 
advanced features (see Section E|). The frame paradigm also facilitates the con- 
struction and adaption of tools, e.g., the OntoEdit and Protege editors and the 
Chimaera integration tool are all being adapted to use OIL/DAML-I-OIL [ 1 211 3| 

m 

On the other hand, basing the language on an underlying mapping to a very 
expressive DL {S'HQ) provides a well defined semantics and a clear understand- 
ing of its formal properties, in particular that the class subsumption/satisfiability 
problem is decidable and has worst case ExpTime complexity m. The mapping 
also provides a mechanism for the provision of practical reasoning services by 
exploiting implemented DL systems, e.g., the FaCT system m 

OIL extends standard frame languages in a number of directions. One of the 
key ideas is that an anonymous class description, or even boolean combinations 
of class descriptions, can occur anywhere that a class name would ordinarily 
be used, e.g., in slot constraints and in the list of superclasses. For example, in 
Figure E (which uses OIL’s “human readable” presentation syntax rather than 
the more verbose RDFS serialisation), a herbivore is described as an animal that 
eats only plants or part-of plants. Points to note are that universally quantified 
(value-type) and existentially quantified (has-value) slot constraints are clearly 
differentiated, and that the constraint on the eats slot is a disjunction, one of 
whose components is an anonymous class description (in this case, just a single 
slot constraint). In addition, it is asserted that the part-of slot is transitive, and 
that its inverse is the slot has-part. Further details of the language will be given 
in Sectional and a complete specification can be found in [Z|. 



slot-def part-of 

subslot-of structural-relation 
inverse has-part 
properties transitive 

class-def defined herbivore 
subclass-of animal 
slot-constraint eats 
value-type plant OR 
slot-constraint part-of has-value plant 



Fig. 1. OIL language example 
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3 OilEd 

OilEd is a simple ontology editor that supports the construction of OIL-based 
ontologies. The basic design has been heavily influenced by similar tools such 
as Protege [El and OntoEdit but OilEd extends these approaches in a 
number of ways, notably through an extension of expressive power and the use 
of a reasoner. 

However, OilEd is not intended as a replacement for such tools — the cur- 
rent implementation of OilEd is intended primarily as a prototype to test and 
demonstrate novel ideas, and compromises have been made in the design and 
implementation. For example, the tool does not provide key functionality for 
collaborative ontology development such as versioning, integration and merging 
of ontologies. Similarly, the powerful tailorability and knowledge acquisition as- 
pects of tools such as Protege have been ignored completely. Rather, the design 
has concentrated on demonstrating how the frame paradigm can be extended to 
deal with a more expressive modelling language, and how reasoning can be used 
to support the design and maintenance of ontologies. 

3.1 OilEd Functionality 

Basic functionality allows the definition and description of classes, slots, individ- 
uals and axioms within an ontology. In general, editing functions are provided 
through graphical means — mouse driven drop down menus, toolbars and but- 
tons. We will not provide a detailed description of the graphical user interface 
here, as it is relatively standard (see Figure Q which provides a screen shot of 
the editors class definition panel). Instead, we will discuss the novel functionality 
offered by the tool. 

Frame Descriptions. The central component used throughout OilEd is the 
notion of a frame description. This consists of a collection of superclasses along 
with a list of slot constraints. This is similar to other frame systems. Where 
OilEd differs, however, is that wherever a class name can appear, a recursively 
defined, anonymous frame description can be used. In addition, arbitrary boolean 
combinations of frames or classes (using and, or and not) can also appear. This 
is in contrast to conventional frame systems, where in general, slot constraints 
and superclasses must be class names. 

As well as being able to assert individuals as slot fillers, several types of 
constraints on slot fillers can be asserted (these kinds of constraint are some- 
times called facets). These include value-type restrictions (all fillers must be of 
a particular class), has-value restrictions (there must be at least one filler of a 
particular class), and explicit cardinality restrictions (e.g., at most three fillers of 
a given class). Each constraint has a clearly defined meaning, removing the con- 
fusion present in some frame systems, where, for example, it is not always clear 
whether the semantics of a slot-constraint should be interpreted as a universal 
or existential quantification. 

Class Definitions. A class definition specifies the class name, along with an 
optional frame description (see above) and a specification of whether the class 



400 



S. Bechhofer et al. 



File Log Export Reasoner Help 



^ 1 ^^ 0 > 



. (^ Classes Nsl Slots 1 Individuals I Axioms • Container 



Classes 

0 company 
0 dog 
0 dog_liker 
0 dog_owner 
0 driver 
0 girl 
0 grownup 
0 haulage_company 
| 0~ haulagejruck_driver _ 
haulage_worker 
0 kid 
0 lorry 
0 lorry_driver 
0 magazine 
0 man 
0 newspaper 
© oidjady 
0 person 
0 pet_owner 
0 publication 
0 quality_broadsheet 



ihaulagejruck_driver 



Properties 




O Primitive i 


S' Defined 



Documentation 




Slot Constraints 



_ type 
|has-value 
ihas-value 



works_for 

drives 



(part_ofhas-value haulage_company] 
truck 



+ X 



D:\OIL tOilEdtontologies'people 



Fig. 2. OilEd Class Panel 



is defined or primitive. If defined, the class is taken to be equivalent to the given 
description (necessary and sufficient conditions). If primitive, the class is taken 
to be an explicit subclass of the given description (necessary conditions) . In the 
specification of the OIL language, classes can have multiple definitions. In OilEd, 
this is disallowed for implementation reasons. Instead classes must have a single 
definition, but the same effect can be achieved through the use of equivalence 
axioms as discussed below. Ontologies using multiple definitions can be read by 
the tool. The first definition encountered will be used as the class definition, 
with any subsequent definitions being translated to the appropriate axioms. 

Slot Definitions. A slot definition gives the name of the slot and allows addi- 
tional properties of the slot to be asserted, e.g., the names of any superslots or 
inverses. If r is a superslot of s, then any two objects related via s must also be 
related via r (i.e., s(a, b) — >■ r(a, b)); if r is an inverse of s, then a is related to b via 
s iff 6 is related to a via r (i.e., s{a,b) O r(6, a)). Domain and range restrictions 
on a slot can also be specified. For example, we can constrain the relationship 
parent to have both domain and range person, asserting that only persons can 
have, and be, parents. As with class descriptions, the domain and range restric- 
tions can be arbitrary class expressions such as anonymous frames or boolean 
combinations of classes or frames, again extending the expressivity of traditional 
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frame editors. Note that in this context, the domain and range restrictions are 
global, and apply to every occurrence of the slot, whether explicit or implicit. 

A slot r can also be asserted to be transitive (i.e., r(a, b) and r(6, c) — f r(a, c)), 
functional (i.e., r(o,6) and r(a,c) — >■ 6 = c) or symmetric (i.e., r{a,b) — f r(6, a)). 

All assertions made about slots are used by the reasoner, and may induce 
hierarchical relationships between classes, e.g., as a result of domain and range 
restrictions. 

Axioms. Another area where the expressive power of OIL/OilEd exceeds that 
of traditional frame languages/editors is in the kinds of axiom that can be used 
to assert facts about classes and their relationships. As well as standard class 
definitions (which are really a restricted form of subsumption/equivalence ax- 
iom), OilEd axioms can also be used to assert the disjointness or equivalence of 
classes (with the expected semantics) along with coverings. A covering asserts 
that every instance of the covered class must also be an instance of at least one 
of the covering classes. In addition, coverings can be said to be disjoint, in which 
case every instance of the covered class must be an instance of exactly one of 
the covering classes. 

Again, these axioms are not restricted to class names, but can involve arbi- 
trary class expressions (anonymous frames or boolean combinations). This is a 
very powerful feature, and is one of the main reasons for the high complexity of 
the underlying decision problem. 

Individuals. Limited functionality is provided to support the introduction and 
description of individuals — the intention within OilEd is that such individuals 
are for use within class descriptions, rather than supporting the production of 
large existential knowledge bases (it is supposed that RDF /RDFS will be used 
directly for this purpose). As an example, we may wish to define the class of 
Italians as being all those Persons who were born in Italy, where Italy is not a 
class but an individual. 

As the FaCT system does not support reasoning with individuals, they are 
treated (for reasoning purposes) as disjoint primitive classes. This is not an ideal 
solution as it does lead to some inferences being lost, in particular those resulting 
from the interaction between individuals and maximum cardinality constraints. 
E.g., it would not be possible to infer that Persons who are citizens of Italy, 
and of no other Country, are citizens of at most one Country. Work is currently 
underway to extend the FaCT reasoner to deal explicitly with such individuals, 
so that complete inference can be provided. 

Concrete Datatypes. Concrete datatypes (string and integers), along with 
expressions concerning concrete datatypes (such as min, max or ranges) can also 
be used within class descriptions. However, the FaCT reasoner does not support 
reasoning over concrete datatypes, and at present OilEd simply ignores concrete 
datatype restrictions when reasoning about ontologies. The theory underyling 
concrete datatypes is, however, well understood cni, and work is also in progress 
to extend the FaCT reasoner with support for concrete datatypes. 
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The latest DAML+OIL language release uses the XML Schema type system 
for the definition of data types. These are not fully supported in our current 
version of OilEd. 

3.2 Reasoning 

In addition to the extended expressivity discussed above, OilEd’s principal nov- 
elty is in its use of reasoning to check class consistency and infer subsumption 
relationships. Reasoning services are currently provided by the FaCT system, 
but in principal any reasoner with the appropriate functionality/connectivity 
could be used. 

FaCT is a DL classifier that offers sound and complete reasoning (satisfiabil- 
ity, subsumption and classification) for two DLs: ST-LJ- and STLQ. FaCT’s most 
interesting features are its expressive logic (in particular the ST-LQ reasoner), its 
optimised tableaux implementation (which has now become the standard for DL 
systems), and its CORE A based client-server architecture [TTij . 

The S'HQ language can completely capture OIL ontologies, with the excep- 
tion of two recently added features: concrete datatypes (strings, numbers, etc.) 
and named individuals in class descriptions. As mentioned above, individuals 
can be dealt with by treating them as pairwise disjoint atomic classes (although 
with some loss of inferential power), while extending FaCT to deal with OIL’s 
concrete datatypes should be relatively straightforward. 

FaCT’s optimisations are specifically aimed at improving the system’s per- 
formance when classifying realistic ontologies. These optimisations lead to per- 
formance improvements of several orders of magnitude when compared with 
older DL and modal logic reasoners, and make the use of reasoning support 
feasable in spite of the discouraging worst case complexity of the underlying de- 
cision problem (ExpTime). The performance improvement is often so great that 
it is impossible to measure precisely as unoptimised systems are virtually non- 
terminating with ontologies that FaCT is easily able to deal with m Taking 
a large medical terminology ontology as an example m, FaCT is able to check 
the consistency of all 2,740 classes and determine the complete class hierarchy 
in about 45 seconds of (700MHz Pentium III) CPU time; unoptimised systems 
have been run for several weeks without their completing even a single class 
consistency test. 

In the current version of OilEd, reasoning is performed on a “single-shot” 
basis, i.e., at some suitable point the user connects to the reasoner and requests 
verification of the ontology. Connection is via FaCT’s COREA based client-server 
interface, which has the advantage that FaCT servers(s) can be running either 
locally or remotely, and can provide a service to many OilEd users. Moreover, 
the FaCT system has reasoning engines for both S'HQ and SHT knowledge 
bases, and if both services are available the user can choose to connect to the 
faster SHT reasoner to verify an ontology that does not include either inverse 
slots or cardinality constraints. The current implementation simply informs the 
user if this is appropriate; future enhancements will include automatic selection 
of an appropriate reasoning service. 
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Fig. 3. Hierarchy pre-classification 



When verification is requested, the ontology is translated into an equivalent 
SV-Q (or SV-T) knowledge base and sent to the reasoner for classification CHI. 
OilEd then queries the classified knowledge base, checking for inconsistent classes 
and implicit subsumption relationships. The results are reported to the user by 
highlighting inconsistent classes and rearranging the class hierarchy display to 
reflect any changes discovered. FaCT/OilEd does not provide any explanation 
of its inferences, although this would clearly be useful in ontology design El- 

Figures 0 and El show the effects of classification on (part of) the hierarchy 
derived from the TAMBIS ontology (see Section EJ- When verifying the on- 
tology, a number of new subsumption relationships are discovered (due to the 
class definitions in the model). In particular we can see that, after verification, 
holoenzyme is not only an enzyme, but also a holoprotein, and that metal-ion and 
small-molecule are both subclasses of cofactor. 

During subsequent editing, changes to the ontology are not communicated 
to the reasoner instantaneously, but only when explicitly requested by the user. 
Future versions of OilEd may incorporate “real-time” reasoning support, but 
the simple interaction model described here was considered appropriate for the 
initial prototype. 
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Fig. 4. Hierarchy post-classification 



3.3 Export 

Although OilEd is primarily intended as an editor for OIL ontologies, the tool 
will export to a number of formats. These include OIL Standard (the “human- 
readable” presentation format for OIL that was used in Figure GJ, OIL-RDFS 
(oil’s standard RDFS serialisation) and DAML-I-OIL (also RDFS). In addition, 
ontologies can be exported as HTML, facilitating viewing of the ontology without 
the tool and class hierarchies generated by the classifier can be exported as 
graphs for viewing with AT&T’s Dotty @ application. 

By exporting ontologies as RDFS, it is envisaged that “RDFS-aware” appli- 
cations will be able to read and interpret OIL ontologies even if they are not 
fully “OIL-aware” . Of course, such applications would be unable to make use of 
all of the information in the model, but may be able to use, for example, the sub- 
class hierarchies within the ontology. In order to facilitate this, OilFd allows the 
possibility of explicitly adding all implicit subsumption relationships to the on- 
tology before export, thus making this information available to non-OIL RDFS 
applications, or even OIL-aware applications that do not employ reasoning. 

^ See http://www.research.att.com/sw/tools/graphviz/ 
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4 Case Study: The TAMBIS Ontology 

The role of ontologies in bioinformatics (the discipline of applying computing 
to molecular biology) has become prominent in the last few years. Ontologies 
are used as a mechanism for expressing and sharing community knowledge, to 
define common vocabularies (e.g., for database annotations), and to support 
intelligent querying over multiple databases j2!Ij. TAMBIS (Transparent Access 
to Multiple Bioinformatic Information Sources) is a mediation system that uses 
an ontology to enable biologists to ask questions over multiple external databases 
using a common query interface. The ontology is central to the TAMBIS system: 
it provides a model that queries can be formed against, it drives the query 
formulation interface, it indexes the middleware wrappers of the component 
sources, and it supports the query rewriting process [21]. The TAMBIS ontology 
(TaO) covers the principal concepts of molecular biology and bioinformatics: 
macromolecules; their motifs, their structure, function, cellular location and the 
processes in which they act. It is an ontology intended for retrieval purposes 
rather than hypothesis generation, so it is broad and shallow rather than deep 
and narrow m- 

The TaO was originally modelled in the DL Grail ini. It was subsequently 
migrated to OIL in order to (a) exploit OIL’s high expressivity so as to maintain 
a better fidelity with biological knowledge as it is currently perceived; (b) use 
reasoning support when building and evolving complex ontologies where the 
knowledge is dynamic and shifting; and (c) be able to deliver the TaO as a 
conventional frame ontology (with all subsumptions made explicit) , thus making 
it accessible to a wider range of (legacy) applications and collaborators. 

The approach to developing the ontology was directly influenced by the range 
of expressivity that OIL affords, and the capabilities of OilEd itself, particularly 
its reasoning facilities. The modelling philosophy was to be descriptive, i.e., to 
model properties and allow as much as possible of the subsumption lattice to be 
inferred by the reasoner. The design methodology was to first construct a basic 
framework of primitive foundation classes and slots, working both top down 
and bottom up, mainly using explicitly stated superclasses. The ontology was 
then incrementally extended and refined by adding new classes, elaborating slot 
fillers and constraints, and “upgrading” to defined classes wherever possible, so 
that class specifications became steadily more detailed and more accurate. This 
process was guided by subsumption reasoning — when elaborating or changing 
classes, the reasoner could be used to check consistency and to show the impact 
on the class hierarchy. 

Figure El shows a (greatly simplified) fragment of the TaO (using OIL’s pre- 
sentation syntax) that we will use to illustrate this methodologylj Originally, 
holoprotein, enzyme and holoenzyme were all primitive classes, with no slot con- 
straints, and an explicitly asserted class hierarchy: holoprotein and enzyme were 
subclasses of protein, and holoenzyme was a subclass of enzyme. During the ex- 
tension and refinement phase, the properties of the various classes were described 

^ The complete ontology can be found at 
http : // img . cs .man. ac .uk/stevens/tambis-oil .html 
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in more detail: it was asserted that a holoprotein binds a prosthetic-group, that 
an enzyme catalyses a reaction, and that a holoenzyme binds a prosthetic-group. 
Several of the classes were also upgraded to being defined when their description 
constituted both necessary and sufficient conditions for class membership, e.g., 
a protein is a holoprotein if and only if it binds a prosthetic-group. This allows 
the reasoner to infer additional subclass relationships w.r.t. holoprotein, and in 
particular that holoenzyme is a subclass of holoprotein. This latter relationship 
probably would have been missed if the ontology had been hand crafted. 



class-def protein 
class-def defined holoprotein 
subclass-of protein 

slot-constraint binds has-value prosthetic-group 
class-def defined enzyme 
subclass-of protein 

slot-constraint catalyses has-value reaction 

class-def defined holoenzyme 
subclass-of enzyme 

slot-constraint binds has-value prosthetic-group 
class-def defined cofactor 

subclass-of (metal-ion or small-molecule) 
disjoint metal-ion small-molecule 

Fig. 5. Simplified fragment of TAMBIS ontology 

The extension and refinement phase also included the addition of axioms 
asserting disjointness, equality and covering, further enhancing the accuracy 
of the model. Referring again to Figure 0 our biologist initially asserted that 
cofactor was a subclass of both metal-ion and small-molecule (a common confu- 
sion over the semantics of ’and’ and ’or’) rather than being either a metal-ion 
or a small-molecule. Subsequently, when it was asserted that metal-ion and 
small-molecule are disjoint, the reasoner inferred that cofactor was logically in- 
consistent, and the mistake was rectified. Modelling mistakes such as these litter 
bioontologies crafted by hand. 

Other advantages derived from the use of OilEd included: 

— The frame-like look and feel of OilEd, and the frame approach of the OIL 
language, made ontology development much less daunting to our biologist 
than writing S'HQ logic expressions would have been. 

— Clipboard facilities provided by OilEd allowed (parts of) frames to be copied 
and pasted, making it easy to experiment with new definitions and to main- 
tain a consistent modelling style. E.g., coenzymeA-requiring-oxidoreductase 
was built by copying nad-requiring-oxidoreductase and changing the constraint 
on the binds slot from nad to coenzymeA. The reasoner then automatically 
migrated the class from being a subclass of holoenzyme to being a subclass 
of coenzyme-requiring-enzyme. 
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— Class definitions can be as simple as possible yet as complex as necessary. 
Parts of the TaO are simply primitive frames and slots; other parts are very 
elaborate and exploit the full expressive power of the OIL language. 

— In TAMBIS, the ontology is managed by an ontology server that makes 
full use of the class definitions, e.g., to classify user generated query classes. 
However, being able to deliver a static “snapshot” of the ontology in the form 
of an RDFS taxonomy has proved extremely convenient when working with 
collaborators who are building ontologies that are in fact simple taxonomies, 
such as the Gene Ontology 

5 Conclusion 

Ontologies are useful in a range of applications, and will play a pivotal role in the 
Semantic Web, where they will provide a source of precisely defined terms that 
can be communicated across people and applications. Reasoning with respect to 
such terms will be important for both the design and deployment of ontologies. 

We have presented OilEd, an ontology editor that has an easy to use frame 
interface, yet at the same time allows users to exploit the full power of an expres- 
sive web ontology language (OIL/D AML-bOIL). We have also shown how OilEd 
uses reasoning to support ontology design and maintenance, and presented a 
case study illustrating how this facility can be used to develop ontologies that 
describe their domains in more detail and with greater accuracy. 

OilEd is a prototype, designed to test and demonstrate novel ideas, and it 
still lacks many features that would be required of a fully-fledged ontology devel- 
opment environment, e.g., it provides no support for versioning, or for working 
with multiple ontologies. Moreover, the reasoning support provided by the FaCT 
system is incomplete for OIL extended with concrete datatypes and individuals, 
and does not include additional services such as explanation. However, in spite 
of these shortcomings, OilEd is already sufficiently well developed to be a very 
useful tool, and to demonstrate the utility of OIL’s integration of features from 
frame, DL and web languages. 
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Abstract. This paper discusses experiments with an agent oriented ap- 
proach to automated and interactive reasoning. The approach combines 
ideas from two subfields of AI (theorem proving/proof planning and 
multi-agent systems) and makes use of state of the art distribution tech- 
niques to decentralise and spread its reasoning agents over the internet. 
It particularly supports cooperative proofs between reasoning systems 
which are strong in different application areas, e.g., higher-order and 
first-order theorem provers and computer algebra systems. 



1 Introduction 

The last decade has seen a development of various reasoning systems which 
are specialised in specific problem domains. Theorem proving contests, such as 
the annual cascQ competition, have shown that these systems typically perform 
well in particular niches but often do poorly in others. First-order provers, for in- 
stance, are not even applicable to higher-order problem formulations. Computer 
algebra systems and deduction systems typically have orthogonal strengths. 
Whereas many hard-wired integrations of reasoning systems have been shown to 
be fruitful, rather few architectures have been discussed so far that try to extend 
the application range of reasoning systems by a flexible integration of a variety 
of specialist systems. 

This paper discusses the implementation of experiments with an agent ori- 
ented reasoning approach, which has been presented as a first idea in pj,IKS99j . 
The system combines different reasoning components such as specialised higher- 
order and first-order theorem provers, model generators, and computer algebra 
systems. It employs a classical natural deduction calculus in the background to 
bridge gaps between sub-proofs of the single components as well as to guarantee 
correctness of constructed proofs. The long term goal is to widen the range of 
mechanisable mathematics by allowing a flexible cooperation between specialist 

^ CADE ATP System Competitions, see also http://www.cs.jcu.edu.au/~-Qtptp/, 

F. Baader, G. Brewka, and T. Eiter (Eds.): KI 2001, LNAI 2174, pp. 409-E^3 2001. 
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systems. This seems to be best achieved by an agent-based approach for a number 
of reasons. Firstly, from a software engineering point of view it offers a flexible 
way to integrate systems. Secondly, and more importantly, the agent-oriented 
approach enables a flexible proof search. This means that each single system - 
in form of a pro-active (software) agent - can focus on parts of the problem it is 
good at, without the need to specify a priori a hierarchy of calls. Currently we 
still work with a centralised approach and focus on the construction of a single 
proof object. This means all agents pick up and investigate the central proof 
object, given in higher-order natural deduction style with additional facilities 
to abstract from pure calculus layer inssDi- In case they find that they are ap- 
plicable in the current proof context they fulfill their task by invoking a tactic 
by, for instance, calling the external system they encapsulate. After consuming 
the available resources they come back and make bids in terms of (probably) 
modified proof objects. Based on heuristic criteri£0one bid is accepted and exe- 
cuted by the central system while the remaining ones are stored for backtracking 
purposes. In this sense global cooperation and communication is established in 
our approach via a central proof object. The benefit is that we have to care 
only about translations into one single proof representation language, which re- 
duces the proof theoretical and logical issues to be addressed. Furthermore, our 
central proof object makes use of a human oriented natural deduction format 
which eases user interaction. For human oriented proof presentation we employ 
the graphical user interface Loui jSHB+flfi] and the proof verbalisation system 
P.rex [Fieinj. 

However, extensive communication amongst the agents is currently also a 
weakness of our system, since too much of the resources are spent on communi- 
cation. Hence, a future goal is to subsequently reduce this overhead by extending 
the agents’ reasoning capabilities and also by decentralising the approach. A dis- 
cussion of particular agenthood aspects of our agents will be given in Section El 

Using the agent paradigm enables us to overcome many limitations of static 
and hard- wired integrations. Furthermore, the agent based framework helps us 
to desequentialise and distribute conceptually independent reasoning processes 
as much as possible. An advantage over hard-wired integrations or even re- 
implementations of specialised reasoners is that it makes the reuse of existing 
systems possible (even without the need for a local installation of these systems) . 
Accessing external systems is orchestrated by packages like Mathweb IFH.T+991 
or the logic broker architecture {sznu . From the perspective of these infrastruc- 
ture packages our work can be seen as an attempt to make strong use of their 
system distribution features. 

Our system currently uses about one hundred agents. They are split in sev- 
eral agent societies where each society is associated with one natural deduction 
rule/tactic of the base calculus. This agent set is extended by further agents 
encapsulating external reasoners. The encapsulation may be a direct one in case 
of locally installed external systems, or an indirect one via the Mathweb frame- 
work, which facilitates their distribution over the internet. Employing numerous 

For instance, bids with closed (sub)goals are preferred over partial results, and big 

steps in the search space are preferred over calculus level steps. 
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Fig. 1. System architecture. 

agents, amongst them powerful theorem provers which are computationally ex- 
pensive, requires sufficient computation resources. Hence, it is crucial to build 
the whole system in a customisable and resource adaptive way. The former is 
achieved by providing a declarative agent specification language and mechanisms 
supporting the definition, addition, or deletion of reasoning agents (as well as 
some other proof search critical components and heuristics) even at run-time. 
For the latter, the agents in our framework can monitor their own performance, 
can adapt their capabilities, and can communicate to the rest of the system 
their corresponding resource information. This enables explicit (albeit currently 
still rudimentary) resource reasoning, facilitated by a specialised resource agent, 
and provides the basic structures for resource adaptive theorem proving. Further 
details on the resource and adaptation aspects are addressed in 1118991 . 

The rest of the paper is structured as follows: Section presents the main 
components of the system architecture. Experiments with the architecture are 
sketched in Section 0 In Sectional we provide an overview of the features of our 
approach and discuss related work. A conclusion/outlook is given in Section 0 



2 System Architecture 

The architecture of our system is depicted in Fig. 0 The core of the system 
is written in Allegro Common Lisp and employs its multi-processing facilities. 
The choice of Common Lisp is due the fact that Omega, our base system, is 
implemented in this programming language; conceptually it can be implemented 
in any multi-processing framework. 

Initial problems, partial proofs as well as completed proofs are represented 
in the Proof Data Structure iTFmi and the n atural deduction infrastructure 
provided by the core system. Omega |BCF+97| . 

Our approach builds on the Reactive Suggestion Mechanism Oants 
as a reactive, resource adaptive basis layer of our framework. Triggered 
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by changes in the proof data structure this mechanism dynamically computes 
applicable commands with their particular parameter instantiations and calls 
external reasoners into the current proof state. An important aspect is that all 
agent computations in this mechanism are de-sequentialised and distributed. 
The idea of this reactive layer is to receive results of inexpensive computations 
(e.g., the applicability of natural deduction rules) quickly while external rea- 
soners search for their respective proof steps within the limits of their available 
resources, until a suggested command is selected and executed. A special re- 
source agent receives performance data from the agents, which monitor their 
own performance, in order to adjust the system at run time. Heuristic criteria 
are used to dynamically filter and sort the list of generated suggestions. They are 
then passed to the selector and/or the user. We give here some sensible heuristic 
criteria. Does a suggestion close a subgoal? Is a subgoal reduced to an essen- 
tially simpler context (e.g., reduction of higher-order problems to first-order or 
propositional logic)? Does a suggestion represent a big step in the search tree 
(proof tactics/methods) or a small step (base calculus rules)? Is the suggestion 
goal directed? How many new subgoals are introduced? 

Agents as well as heuristic criteria can be added/deleted/modified at run 
time. Due to lack of space Oants cannot be described here in detail; for this we 
refer the reader to Esnii- 

Oants provides agents that do computations on the basic natural deduction 
calculus. It also provides agents that invoke additional proof tactics/methods 
and external reasoning systems. The external reasoning systems are called by the 
agent-shells indirectly via the Mathweb system. That is, the agents themselves 
are realised as concurrent Lisp processes in the core system. These processes ac- 
tivate themselves and make calls to Mathweb services when their applicability 
criteria are fulfilled (this contrasts calls by human users to external systems in 
interactive proof environments). 

We extended the approach from in the context of our work to integrate 

partial proofs as results from the external reasoning systems into the overall proof 
as well as to store different alternative subproofs simultaneously. Moreover, we 
extended Omega’s graphical user interface Loui to be able to display different 
subproofs of external reasoners as choices for the user. 

The Mathweb system realises calls to external reasoners which may be dis- 
tributed over the internet. In our most recent experiments we extensively tested 
the new One-Mathweb system which is based on a multi-broker architecture. 
Each broker has knowledge about its directly accessible reasoning systems, and 
also about urls to other One-Mathweb brokers on the internet. For example, 
in our experiments the reasoning agents gained access to the computer alge- 
bra system Maple running in Saarbriicken. For this we simply had to inform 
the Birmingham Mathweb broker (which for license reasons cannot offer a 
Maple service locally) about the existence and url of the Saarbriicken broker. 
The Saarbriicken broker then connects the Birmingham broker (which receives 
and answers to the requests of the reasoning agents) with the Maple service. 
Currently our system links up with the computer algebra systems Maple and 
Gap running in Saarbriicken, and locally with the higher-order theorem provers 
Leo and Tps, the first-order theorem prover Otter (employed also as our 
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propositional logic specialist), and Satchmo (employed as a model generator). 
Mathweb is described in detail in fFH.T+99) . 

Once the reactive suggestion mechanism dynamically updates and heuris- 
tically sorts the list of suggestions, which are commands together with their 
particular parameter instantiations, it passes the list on to the selector. Its 
main task is to automatically execute the heuristically preferred command, and 
hence, initiate an update of the proof data structure. Furthermore, the selector 
stores the non-optimal, alternative command suggestions in a special store. The 
information in this store is used when backtracking to a previous state in the 
proof data structure becomes necessary. Instead of a complete initialisation the 
reactive suggestion mechanism is then simply initialised with the already com- 
puted backtracking information for the current proof context. Backtracking is 
caused when the reactive layer produces no suggestions or when a user defined 
maximal deptlH in the proof data structure is reached. 

The backtrack store maintains backtracking information for the proof data 
structure. This information includes representations of the suggestion computa- 
tions that have been previously computed but not executed. Additionally the 
store maintains the results of external system calls modulo their translation in 
the core natural deduction calculus. That is, the immediate translation of exter- 
nal system results is also done by the reactive suggestion layer, and the results 
of these computations are memorised for backtracking purposes as well. If the 
system or the user selects to apply the result of an external system, the proof 
data structure is updated with the translated proof object. Future work will in- 
clude investigating whether the backtrack store should be merged with the proof 
data structure. The idea is that each single node in a proof directly maintains 
its backtracking alternatives instead of using an indirect maintenance via the 
backtracking store. 

The tasks of the user interface in our framework are: 

1. To visualise the current proof data structure and to ease interactive proof 
construction. For this purpose we employ Omega’s graphical user interface 
Loui |SHB+99| . 

2. To dynamically present to the user the set of suggestions, which pop up 
from the reactive layer to the user, and to provide support for analysing 
or executing them. This is realised by structured and dynamically updated 
pop-up windows in Loue 

3. To provide graphical support for analysing the results of external systems, 
that is, to display their results after translation/representation in the proof 
data structure. We achieve this by extending Loui so that it can switch 
between the global proof data structure and locally offered results by external 
systems. 

4. To support the user in interacting with the automated mechanism and in 
customising agent societies at run-time. 

From an abstract perspective, our system realises proof construction by going 
through a cycle which consists of assessing the state of the proof search process, 

® Iterative deepening proof search wrt. to the maximal depth is conceptnally feasible 
but not realised yet. 
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evaluating the progress, choosing a promising direction for further search and 
redistributing the available resources accordingly. If the current search direction 
becomes increasingly less promising then backtracking to previous points in the 
search space is possible. Only successful or promising proof attempts are allowed 
to continue searching for a proof. This process is repeated until a proof is found, 
or some other terminating condition is reached. 

3 Experiments 

In this section we report on experiments we conducted with our system to demon- 
strate the usefulness of a flexible combination of different specialised reasoning 
systems. Among others we examined different problem classes: 

1. Set examples which demonstrate a cooperation between higher-order and 
first-order theorem provers. For instance, prove: 

Vx, y,z.{x = yUz)-^{yCxAzCxA Vu. {y Q v A z Q v) ^ {x Q v) 

2. Set equations whose validity/invalidity is decided in an interplay of a natu- 
ral deduction calculus with a propositional logic theorem prover and model 
generator. For instance, prove or refute: 

a) Vx, y,z.{xVJ y) C\ z = {xC\ z)VJ {y C\ z) 

b) Vx, y, z. (x U y) n z = (x U z) n (y U z) 

3. Concrete examples about sets over naturals where a cooperation with a 
computer algebra system is required. For instance {gcd and Icm stand for 
the ‘greatest common divisor’ and the ‘least common multiple’): 

{x\x > gcd(10, 8) A a: < Zcm(10, 8)} = {x\x < 40} fl {x\x > 2} 

This set is represented by the lambda expression 

{Xx.x > gcd{10, 8) A X < lcm{10, 8)) = {Xx.x < 40) fl {Xx.x > 2) 

4. Examples from group theory and algebra for which a goal directed natural 
deduction proof search is employed in cooperation with higher-order and 
first-order specialists to prove equivalence and uniqueness statements. These 
are for instance of the form 

\3o .Group{G,o)\ <t4> [3*.Monoid(M,*) A Inz;erses(M, *, C/mf(M, *))] 

Here Group and Monoid refers to a definition of a group and a monoid, 
respectively. Inverses{M,-k,Unit{M,-k)) is a predicate stating that every 
element of M has an inverse element with respect to the operation * and 
the identity Unit{M,-k). Unit{M,-k) itself is a way to refer to that unique 
element of M that has the identity property. 

We will sketch in the following how the problem classes are tackled in our system 
in general and how the proofs of the concrete examples work in particular. 

3.1 Set Examples 

The first type of examples is motivated by the shortcomings of existing higher- 
order theorem provers in first-order reasoning. For our experiments we used 
the Leo system [HK98] . a higher-order resolution prover, which specialises in 
extensionality reasoning and is particularly successful in reasoning about sets. 



Experiments with an Agent- Oriented Reasoning System 415 




Fig. 2. Agent based cooperation between Leo and Otter. 



Initialised with a set problem Leo tries to apply extensionality reasoning in 
a goal directed way. On an initial set of higher-order clauses, it often quickly 
derives a corresponding set of essentially first-order clauses0 Depending on the 
number of generated first-order and other higher-order clauses Leo may get 
stuck in its reasoning process, although the subset of first-order clauses could be 
easily refuted by a first-order specialist. 

For our examples the cooperation between Leo and the first-order specialist 
Otter works as depicted in Fig. Q The initial problem representation in the 
proof data structure is described in Part 1 of Fig. El The initialisation triggers 
the agents of the reactive suggestion layer which start their computations in 
order to produce suggestions for the next proof step. 

The agent working for Leo first checks if there is any information from the 
resource agent that indicates that Leo should stay passive. If not, it checks 
whether the goal C is suitable for Leo by testing if it is a higher-order problem. 
In case the problem is higher-order the agent passes the initial problem consisting 
of the goal C and the assumptions Pi, . . . ,P„ to Leo. While working on the 
input problem (as indicated by the shaded oval in Part 2 of Fig. 0 ) Leo derives 
(among others) various essentially first-order clauses (e.g., FOi...FO„). For 
the particular type of cooperation described here, it is important that after a 
while this subset becomes large enough to be independently refutable. If after 
consuming all the resources made available by the reactive suggestion layer Leo 
still fails to deliver a completed proof, it then offers a partial proof consisting of 
a subset of first-order and essentially first-order clauses (after translation into 
prenex normal form, e.g., VS-FO^ A . . . AFO(j, where the FO' are disjunctions of 
the literals of FOi and x stands for the sequence of all free variables in the scope) . 
In case Leo’s suggestion wins over the suggestions computed by other agents, 
its partial result is represented in the proof data structure and the reactive 
suggestion mechanism is immediately triggered again to compute a suggestion 
for the next possible proof step. Since Leo’s partial result is now the new subgoal 
of the partial proof, first-order agents, like the one working for Otter, can pick 
it up and ask Otter to prove it (see Part 3 of Fig. EJ. If Otter signals a 

^ By essentially first-order we mean a clause set that can be tackled by first-order 
methods. It may still contain higher-order variables, though. 
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successful proof attempt before consuming all its given resources, its resolution 
proof is passed to the natural deduction translation module Tramp jMeiOOL 
which transforms it into a proper natural deduction proof on an assertion levelo 
We experimented with 121 simple examples, that is, examples that can be 
automatically proved by Leo alone. The results showed that the command execu- 
tion interval chosen by the selector is crucial, since it determines the computation 
time ct made available to the external systems. 

— If ct is sufficiently high, then the problem is automatically proved by Leo 
( in case of simple examples that can be solved by Leo alone). 

— If ct is not sufficient for Leo to come up with a proof, but still enough to pro- 
duce a refutable subset of essentially first-order clauses, then a cooperative 
proof is constructed as described above. 

— If ct is not sufficient to even guarantee a subset of refutable essentially first- 
order clauses, then the problem is tackled purely on natural deduction level, 
however not necessarily successfully. 

We also solved several examples which cannot be solved with Leo alone. One 
of them is the concrete example given above, which, to our knowledge, cannot 
be easily solved by a single automated theorem prover. In our experiments, 
Leo alone ran out of memory for the above problem formulation, and Otter 
alone could not find a proof after running 24 hours in auto mode on a first- 
order formulation of the problem. Of course, an appropriate reformulation of the 
problem can make it simple for systems like OTTERto prove this new formulation. 



3.2 Set Equations 

The second type of set examples illustrates a cooperation between automated 
natural deduction agents, a propositional prover and a model generator. The 
proofs follow a well-known set theoretic proof principle: they are constructed first 
by application of simple natural deduction agents that reduce the set equations 
by applying set extensionality and definition expansion to a propositional logic 
statement. This statement is then picked up by an agent working for a propo- 
sitional logic prover (here we again use Otter encapsulated in another agent 
shell with a slightly modified applicability check and a different representation 
translation approach) and a counter-example agent which employs Satchmo. 
The logic statement is then either proved or refuted. Thus, valid and invalid 
statements are tackled analogously in all but the last step. 

In case ffial of our concrete examples several V/ (universal quantification 
introduction in backward reasoning) applications introduce (aU&)ric = (aric)U 
(5 n c) as new open subgoal. Set extensionality gives us Vm . u G {aUb)(lc^ u G 
((aflc) U (6ric)). A further V/ application and subsequent definition expansions 
(where aUb := Xz. {z G a) \/ {z G b), a (1 b := Xz. (z G a) A (z G b), and u G a := 
a{u)) reduce this goal finally to {a{d) V b{d)) A c{d) = (a{d) A c{d)) V (b{d) A c{d)) 

® While Tramp already supports the transformation of various machine oriented first- 
order proof formats, further work will include its extension to higher-order logic, such 
that also the proof step justified in Fig. 0 with ‘LEO-derivation’ can be properly 
expanded into a verifiable natural deduction proof. 
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which contains no variables and which is a trivial task for any propositional 
logic prover. In case 1 21)11 we analogously derive {a{d) V b{d)) A c{d) = (a(d) V 
c{d)) A {b{d) V c{d)), but now a model generator agents presents the counter- 
model a{d),b{d),-'c{d). That is, it points to the set of all d such that d G a, 
d G b, but d ^ c. Hence, the model generator comes up with a counter-example 
to the expression in fl2 hll . 

We have experimented with an automatically and systematically generated 
testbed consisting of possible set equations involving fl, U, set-minus operations 
up to nesting depth of 5 in maximally 5 variables. We classified 10000 examples 
with our system discovering 988 correct and 9012 false statements. Naturally, 
the correct statements are probably also solvable with the cooperation of Leo 
and Otter. 



3.3 Examples with Computer Algebra 

The next type of examples has cross-domain character and requires a combi- 
nation of domain specific systems. In order to tackle them we added a sim- 
plification agent which links the computer algebra system Maple to our core 
system. As an application condition this agent checks whether the current sub- 
goal contains certain simplifiable expressions. If so, then it simplifies the sub- 
goal by sending the simplifiable subterms (e.g., x > (7cd(10,8)) via Math- 
web to Maple and replaces them with the corresponding simplified terms 
(e.g., X > 40). Hence, the new subgoal suggested by the simplification agent 
is: {\x.x > 2 t\ X < 40) = {\x.x < 40) fl {Xx.x > 2). Since no other agent 
comes up with a better alternative, this suggestion is immediately selected and 
executed. Subsequently, the Leo agent successfully attacks the new goal after 
expanding the definition of fl. We have successfully solved 50 problems of the 
given type and intend to generate a large testbed next. 



3.4 Group Theory and Algebra Examples 

The group theory and algebra examples we examined are rather easy from a 
mathematical viewpoint, however, can become non-trivial when painstakingly 
formalised. An example are proofs in which particular elements of one math- 
ematical structure have to be identified by their properties and transferred to 
their appropriate counterparts in an enriched structure. The equivalence state- 
ment given above in 0) where the unit element of the monoid has to be identified 
with the appropriate element of the group are in this category. In higher-order 
this can be done most elegantly using the description operator t (cf. !And72) 
for description in higher-order logics) by assigning to the element in the group 
the unique element in the monoid that has exactly the same properties. In the 
context of our examples we employed description to encode concepts like the 
(unique) unit element of a group by a single term that locally embodies the par- 
ticular properties of the encoded concept itself. If properties of the unit element 
are required in a proof then the description operator has to be unfolded (by 
applying a tactic in the system) and a uniqueness subproof has to carried out. 
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However, an open problem is to avoid unnecessary unfoldings of the description 
operator as this may overwhelm the proof context with unneeded information. 

The idea of the proofs is to divide the problems into smaller chunks that 
can be solved by automated theorems provers and if necessary to deal with for- 
mulae involving description. The ND search procedure implemented in Oants 
has the task to successively simplify the given formulae by expanding defini- 
tions and applying ND inferences. After each proof step the provers try to solve 
the introduced subproblems. If they all fail within the given time bound the 
system proceeds with the alternative ND inferences. The quantifier rules intro- 
duce Skolem variables and functions when eliminating quantifications. These 
are constrained either by the application of a generalised Weaken rule, using 
higher-order unification, or by the successful solution of subproblems by one 
of the provers, which gives us the necessary instantiation. Problems involving 
higher-order variables (for which real higher-order instantiations are required) 
can generally not be solved (in this representation) by first-order provers. How- 
ever, once an appropriate instantiation for the variables has been computed a 
first-order prover can be applied to solve the remaining subproblems. Substitu- 
tions for introduced Skolem variables are added only as constraints to the proof, 
which can be backtracked if necessary. 

When a point is reached during the proof where neither applicable rules nor 
solutions from the provers are available, but the description operator still occurs 
in the considered problem, two theorems are applied to eliminate description. 
This results in generally very large formulae, which can then again be tackled 
with the ND rules and the theorem provers. 

In our experiments with algebra problems we have successfully solved 20 
examples of the described type. 



Our experiments show that the cooperation between different kinds of reason- 
ing systems can fruitfully combine their different strengths and even out their 
respective weaknesses. In particular, we were able to successfully employ Leo’s 
extensionality reasoning with Otter’s strength in refuting large sets of first- 
order clauses. Likewise, our distributed architecture enables us to exploit the 
computational strength of Maple in our examples remotely over the internet. 
As particularly demonstrated by the last example class the strengths of external 
systems can be sensibly combined with domain specific tactics and methods, and 
natural deduction proof search. 

Note that our approach does not only allow the combination of heterogeneous 
systems to prove a problem, but it also enables the use of systems with opposing 
goals in the same framework. In our examples the theorem prover and the model 
generator work in parallel to decide the validity of the current (propositional) 
goal. 

Although many of our examples deal with problems in set theory they already 
show that the cooperation of differently specialised reasoning systems enhances 
the strengths of automated reasoning. The results also encourage the applica- 
tion of our system to other areas in mathematics in the future. However, there 
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is a bottleneck for obtaining large proofs, namely the translation between the 
different systems involved, in particular, in the presence of large clause sets. 

4 Discussion 

Our work is related to blackboard and multi-agent systems in general, and to 
approaches to distributed proof search and agent-oriented theorem proving in 
particular. Consequently, the list of related work is rather long and we can men- 
tion only some of it. We first summarise different facets of our approach which 
we then use to clarify the differences to other approaches and to motivate our 
system design objectives. Our system: 

(1) aims to provide a cognitively adequate assistant tool to interactively and/or 
automatically develop mathematical proofs; 

(2) supports interaction and automation simultaneously and integrates reactive 
and deliberative proof search; 

(3) maintains a global proof object in an expressive higher-order language in 
which results of external systems can be represented; 

(4) employs tools as Loui ISHB+QOl or P.rex It i^-iill to visualise and verbalise 
proofs, i.e., communicate them on a human oriented representation layer; 

(5) couples heterogeneous external systems with domain specific tactics and 
methods and natural deduction proof search; i.e., our notion of heterogene- 
ity comprises machine oriented theorem proving as well as tactical theorem 
proving/proof planning, model generation, and symbolic computation; 

(6) reuses existing reasoning systems and distributes them via Mathweb (In or- 
der to add a new system provided by Mathweb the user has to: a) provide 
an abstract inference step/command modelling a call to the external rea- 
soner, b) define the parameter agents working for it, and c) (optional) adapt 
the heuristic criteria employed by the system to rank suggestions. Due to 
the declarative agent and heuristics specification framework these steps can 
be performed at run time.); 

(7) supports competition (e.g., proof versus countermodel search) as well as 
cooperation (e.g., exchange of partial results); 

(8) follows a skeptical approach and generally assumes that results of exter- 
nal reasoning system are translated in the central proof object (by employ- 
ing transformation tools such as Tramp jMeiflflj l where they can be proof- 
checked; 

(9) employs resource management techniques for guidance; 

(10) supports user adaptation by enabling users to specify/modify their own con- 
figurations of reasoning agents at run-time, and to add new domain specific 
tactics and methods when examining new mathematical problem domains; 

(11) stores interesting suboptimal suggestions in a backtracking stack and sup- 
ports backtracking to previously dismissed search directions; 

(12) supports parallelisation of reasoning processes on different layers: term-level 
parallelisation is achieved by various parameter agents of the commands/ 
abstract inferences, inference-level parallelisation is supported by the ability 
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to define new powerful abstract inferences which replace several low level in- 
ferences by a single step (a feature inherited from the integrated tactical the- 
orem proving paradigm), and proof-search- level parallelisation is supported 
by the competing reasoning systems. 

Taken individually none of the above ideas is completely new and for each 
of these aspects there exists related work in the literature. However, it is the 
combination of the above ideas that makes our project unique and ambitious. 

A taxonomy of parallel and distributed (first-order) theorem proving systems 
is given in fTCTT] . As stated in (O), our approach addresses all three classi- 
fication criteria introduced there: parallelisation on term, inference, and search 
level. However, full or-parallelisation is not addressed in our approach yet. This 
will be future work. 

A very related system is the Techs approach which realises a cooper- 

ation between a set of heterogeneous first-order theorem provers. Partial results 
in this approach are exchanged between the different theorem provers in form of 
clauses, and different referees filter the communication at the sender and receiver 
side. This system clearly demonstrates that the capabilities of the joint system 
are bigger than those of the individual systems. Techs’ notion of heterogeneous 
systems, cf. (0 above, however, is restricted to a first-order context only. Also 
symbolic computation is not addressed. Techs mm and its even less heteroge- 
neous predecessors Teamwork pK96j and Discount [ADF95j are much more 
machine oriented and less ambitious in the sense of aspects drj~0. However, 
the degree of exchanged information (single clauses) in all these approaches is 
higher than in our centralised approach. Unlike in the above mentioned systems, 
our interest in cooperation, however, is in the first place not at clause level, but 
on subproblem level, where the subproblem structure is maintained by the cen- 
tral natural deduction proof object. Future work includes investigating to what 
extend our approach can be decentralised, for instance, in the sense of Techs, 
while preserving a central global proof object. 

In contrast to many other approaches we are interested in a fully skeptical 
approach, cf. (0 and the results of some external reasoners (e.g., for Otter 
Tps, and partially for computer algebra systems) can already be expanded and 
proof checked by translation in the core natural deduction calculus. However, for 
some external systems (e.g., Leo) the respective transformation tools still have 
to be provided. While they are missing, the results of these system, modelled as 
abstract inferences in natural deduction style, cannot be expanded. 

Interaction and automation are addressed by the combination of Ilf & 
Techs EM- With respect to aspects ®-(EJ, especially m, there are vari- 
ous essential differences in our approach. The design objectives of our system are 
strongly influenced by the idea to maintain a central proof object which is ma- 
nipulated by the cooperating and competing reasoning agents, and mirrors the 
proof progress. This central natural deduction proof object especially eases user 
interaction on a human oriented layer, cf. Q and 0, and supports skepticism 
as described above. In some sense, external systems are modelled as new proof 
tactics. Extending the background calculus and communication between them 
is currently only supported via the system of blackboards associated with the 
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current focus of the central proof object. This relieves us from addressing logical 
issues in the combination of reasoning systems at the proof search layer. They 
are subordinated and only come into play when establishing the soundness of 
contributions of external reasoners by expanding their results on natural deduc- 
tion layer. A centralised approach has advantages in the sense that it keeps the 
integration of n heterogeneous systems, with probably different logical contexts, 
simple and it only requires n different proof (or result) transformation tools 
to natural deduction arguments. In particular the overall proof construction is 
controlled purely at the natural deduction layer. 

However, experiments indicated that aside from these advantages, the bottle- 
neck of the system currently is the inefficiency in the cooperation of some external 
systems, especially of homogeneous systems specialised in resolution style prov- 
ing which cannot directly communicate with each other. Future work therefore 
includes investigating whether the approach can be further decentralised with- 
out giving up much of the simplicity and transparency of the current centralised 
approach. 

With the centralisation idea, we adopted a blackboard architecture and our 
reasoning agents are knowledge sources of it. In the terminology of |Weihh| our 
reasoning agents can be classified as reactive, autonomous, pro-active, coop- 
erative and competitive, resource adapted, and distributed entities. They, for 
instance, still lack fully deliberative planning layers and social abilities such 
as means of explicit negotiation (e.g., agent societies are defined by the user 
in Oants and, as yet, not formed dynamically at run-time IBHOU). In this 
sense, they are more closely related to the Hasp |1NI*AR.83) or POLIGON |R,ic8H| 
knowledge sources than to advanced layered agent architectures like Inter- 
rap pil^ . However, in future developments a more decentralised proof search 
will make it necessary to extend the agenthood aspects in order to enable agents 
to dynamically form clusters for cooperation and to negotiate about efficient 
communication languages. 

5 Conclusion 

In this paper we presented our agent-based reasoning system. Our framework is 
based on concurrent suggestion agents working for natural deduction rules, tac- 
tics, methods, and specialised external reasoning systems. The suggestions by the 
agents are evaluated after they are translated into a uniform data representa- 
tion, and the most promising direction is chosen for execution. The alternatives 
are stored for backtracking. The system supports customisation and resource 
adapted and adaptive proof search behaviour. 

The main motivation is to develop a powerful and extendible system for 
tackling, for instance, cross domain examples, which require a combination of 
reasoning techniques with strengths in individual domains. However, our moti- 
vation is not to outperform specialised systems in their particular niches. The 
agent paradigm was chosen to enable a more flexible integration approach, and 
to overcome some of the limitations of hardwired integrations (for instance, the 
brittleness of traditional proof planning where external systems are typically 
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called within the body of proof methods and typically do not cooperate very 
flexibly) . 

A cognitive motivation for a flexible integration framework presented in this 
paper is given from the perspective of mathematics and engineering. Depending 
on the specific nature of a challenging problem, different specialists may have 
to cooperate and bring in their expertise to fruitfully tackle a problem. Even 
a single mathematician possesses a large repertoire of often very specialised 
reasoning and problem solving techniques. But instead of applying them in a 
fixed structure, a mathematician uses own experience and intuition to flexibly 
combine them in an appropriate way. 

The experience of the project points to different lines of future research. 
Firstly, the agent approach offers an interesting framework for combining auto- 
mated and interactive theorem proving on a user-oriented representation level 
(and in this sense it differs a lot from the mainly machine-oriented related work). 
This approach can be further improved by developing a more distributed view 
of proof construction and a dynamic configuration of cooperating agents. Sec- 
ondly, in order to concurrently follow different lines of search (or-parallelism) , a 
more sophisticated resource handling should be added to the system. Thirdly, 
the communication overhead for obtaining large proofs is the main performance 
bottleneck. More efficient communication facilities between the different systems 
involved have to be developed. Contrasting the idea of having filters as suggested 
in fTiTbil we also want to investigate whether in our context (expressive higher- 
order language) abstraction techniques can be employed to compress the ex- 
changed information (humans do not exchange clauses) during the construction 
of proofs. 

Further future work includes improving several technical aspects of the cur- 
rent Omega environment and the prototype implementation of our system that 
have been uncovered during our experiments. We would also like to test the 
system in a real multi-processor environment, where even the agent-shells for 
external reasoners can be physically distributed - currently, the agent-shells, 
which are local, make indirect calls (via Mathweb) to the external systems. 
Furthermore, we will integrate additional systems and provide further repre- 
sentation translation packages. The agents’ self-monitoring and self-evaluation 
criteria, and the system’s resource adjustment capabilities will be improved in 
the future. We would also like to employ counter-example agents as indicators 
for early backtracking. Finally, we need to examine whether our system could 
benefit from a dynamic agent grouping approach as described in [KW95j , or from 
an integration of proof critics as discussed in inmi . 
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Abstract. Most state-of-the-art navigation systems for autonomous service robots 
decompose navigation into global navigation planning and local reactive naviga- 
tion. While the methods for navigation planning and local navigation are well 
understood, the plan execution problem, the problem of how to generate and pa- 
rameterize local navigation tasks from a given navigation plan, is largely unsolved. 
This article describes how a robot can autonomously learn to execute navigation 
plans. We formalize the problem as a Markov Decision Problem (mdp), discuss 
how it can be simplified to make its solution feasible, and describe how the robot 
can acquire the necessary action models. We show, both in simulation and on a 
RWI B21 mobile robot, that the learned models are able to produce competent 
navigation behavior. 



1 Introduction 

Robot navigation is the task of reliably and quickly navigating to specified locations in 
the robot’s operating environment. Most state-of-the-art navigation systems for mobile 
service robots consist - besides components for map learning and the estimation of 
the robot’s position - of components for global navigation planning and local reactive 
navigation IKBM981 . 

Using a map of the operating environment, global navigation planning computes 
plans for navigating to specified locations in the environment. Navigation plans are typ- 
ically sequences of discrete actions, mappings from robot states into discrete navigation 
actions (navigation policies), or paths, sequences of intermediate destinations. Latombe 
IILat91ll gives a comprehensive overview of these algorithms. Approaches to compute 
navigation policies using the mdp framework are described in | KCK96ijSK9.5h’BB+98l . 

Researchers have also investigated a variety of methods for carrying out local reactive 
navigation tasks. These tasks are those in which the destinations are located in the 
surroundings of the robot and for which no global (static) map of the environment is used. 
The reactive navigation methods often employ concurrent, continuous and sensor-driven 
control processes and a behavior arbitration method that continually combines the output 
signals of the individual control processes into common control signals for the robot’s 

* The research reported in this paper is partly funded by the Deutsche Forschungsgemeinschaft 
(DFG) under contract number BE 2200/3-1. 
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drive. Arkin tArlc9gll gives a comprehensive introduction to the principles of (hehavior- 
based) reactive control. Other approaches generate trajectories for local navigation tasks 
based on simple models of the robot dynamics and choose the trajectories with the 
highest utility |FBT97-Sim96l| . 




Fig. 1. Two behavior traces produced by the same navigation plan and local navigation module 
using different plan execution mechanisms. The robot’s position is depicted by a circle where the 
size of the circle is proportional to the robot’s translational speed. 



While the methods for solving these subproblems are well understood, the plan 
execution problem, the problem of how to combine navigation planning and reactive 
navigation to produce competent navigation behavior, is largely unsolved. Figure Q] il- 
lustrates that the generation of local navigation subtasks and parameterizations of local 
navigation processes for a given navigation plan has an enormous impact on the per- 
formance of the robot. The figure depicts two navigation traces that are generated by 
the same navigation plan and reactive navigation module using different plan execution 
schemes. Please refer to tBBOUl for a more detailed explanation. 

Designing the appropriate plan execution mechanism is very hard and therefore the 
mechanism is often implemented in an ad-hoc manner and hand-tuned for different 
applications. Several factors complicate its design. The methods used for navigation 
planning and local navigation are incompatible. Navigation planning is performed as 
open-loop control whereas plan execution performs closed-loop control, planning often 
assumes an abstract discretization of the state and action space whereas reactive naviga- 
tion deals with continuous processes and asynchronously arriving sensor data streams, 
planning often works in a drastically reduced state space that ignores dynamic obstacles 
and the robot’s dynamic state, which in turn are handled by reactive navigation. Even 
worse, the appropriateness of a given plan execution mechanism often depends on the 
characteristics of the robot and the specifics of the operating environments. 

These difficulties suggest that the proper plan execution mechanism should be au- 
tonomously learned by the robot. Surprisingly, this computational problem has received 
little attention so far. 



Learning to Execute Navigation Plans 427 



In this paper we describe how a mobile robot can learn to improve its performance by 
improving the interaction between navigation planning and local navigation. As the main 
scientific contribution of this article we will show how the navigation plan execution 
problem can be formulated as a Markov Decision Problem (mdp) in such a way that the 
solutions to the mdp will produce competent navigation behavior and that the information 
needed for solving the mdp can be automatically learned by the robot. We will illustrate 
the power of the approach both in simulation and on a real robot. The implemented 
plan execution component together with the learning component does not necessarily 
form a new layer, but can be integrated either in the reactive navigation layer or the path 
planning layer as done in our implementation. 

The remainder of this article is organized as follows: Section 2 states the navigation 
plan execution problem as a Markov Decision Problem and discusses how this mdp can 
be simplihed to make its solution feasible. In Section 3 we describe how the models 
necessary to compute a policy for the problem can be autonomously learned. In Section 
4 we experimentally demonstrate that the learned action selection policy significantly 
improves the robot’s navigation performance. Section 5 discusses related work. 

2 The Plan Execution Problem as an MDP 

In this section we briefly introduce the notion of Markov Decision Problems (mdps) and 
then formulate the plan execution problem as an mdp. 

2.1 MDPs for Stochastic Domains 

A Markov Decision Process is a general framework for the specification of simple control 
problems where an agent acts in a stochastic environment and receives rewards from its 
environment. A solution of an mdp is a policy, a mapping from states into actions that 
maximizes the expected accumulated reward. 

More formally, an mdp is given by a set of states S, a set of actions A, a probabilistic 
action model P(S'|S', A), and a reward function R. P(s'js,a) denotes the probability 
that action a taken in state s leads to the state s' . The reward function is a mapping 
R : S X A ^ ]R where R{s, a) denotes the immediate reward gained by taking action 
a in state s. The property that the effect of an action only depends on the action and the 
state in which they are executed is called the Markov property. In this formal setting a 
solution of a Markov Decision Problem is a policy tt such that 



7t(s) = argmax[i?(s, a) + 7 f P(s'|s, a)F*(s')] (1) 

aeA J 

s'gS 

The action 7 r(s) in equationQlis the action that maximizes the sum of the immediate 
reward R{s,a) for taking action a in state s and y*(s') the expected future reward after 
executing the action. (s') denotes the utility of being in state s' when acting according 
to the optimal policy. The variable 7 is the discounting factor which weighs expected 
future rewards with respect to how far in the future they will occur. Kaelbling IKLC98II 
gives a detailed discussion of mdps and pomdps, a generalization of mdps where the 
state is not fully observable. 
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2.2 Formulating Plan Execution as an mdp 

For the following discussion we assume that the robot executes navigation plans only 
when it is certain about its location in the environment. This is a realistic assumption as 
we use a localization technique that detects when the robot becomes uncertain about its 
position in the environment, interrupts navigation tasks in order to relocalize itself, and 
continues the the navigation tasks after successful relocalization HBFT97 1 . 

Under this assumption we can consider the plan execution problem as an mdp. 
Unfortunately, the state space of the plan execution problem is inhnite and even under 
reasonable discretizations is the determination of the optimal policy defined by equation^ 
computationally too expensive and the policy itself too large to be stored. Therefore, we 
will heuristically simplify equation[I]and only apply this simplified equation to determine 
the plan execution policy. 



The State Space. For the plan execution problem we characterize states s G S' by the 
robot’s position and orientation, its current sensor readings and the remaining navigation 
plan given by the path p = [pojPIj •■•jPn] from the robot’s current position to its des- 
tination. The latter is essential for the plan execution problem, but as paths of arbitrary 
length in general do not have compact descriptions of hxed length, this also makes the 
problem difficult. 

Ideally the state should also include the dynamic state of the robot, that is its velocity 
and acceleration. In this paper we still abstract away from the dynamic state and postpone 
the incorporation of dynamic states into our next research steps. We compensate for this 
simplihcation by always letting the robot stop before the next action is chosen. 



The Actions. We take the actions that can be executed in a state s to be a point in the re- 
maining navigation plan p = [po,Pi, ■■■,Pra] that can be passed to the reactive navigation 
component as intermediate target point. This choice of possible actions reflects the fol- 
lowing trade-off: While in some circumstances an intermediate target that is farer away 
from the robot’s current position might be advantageous because it allows the reactive 
navigation system for smoother trajectories, in cluttered environments the target points 
have to be closer to the robot to be reachable at all. Please note that we do not address 
the trade-off between short but difficult versus easy but long paths as this is already done 
by the path planning system. For now we also ignore other actions such as setting the 
navigation speed, turning in place, and others. 

Reward and Probabilistic Action Models. The probabilistic action model P{S\S, A) 
is a continuous probability distribution in an infinite space. Because this distribution is 
very complex it is neither possible to specify the distribution by hand nor to learn it 
e.g. using mixture models. Therefore, the distribution has to be approximated. For this 
purpose we will assume that an action a can either be successful, in this case the robot 
will be where it intended to be after executing a, or unsuccessful. In the latter case we 
assume the robot to be where it started to execute action a. We consider an action a to be 
unsuccessful when the robot has not reached its target point within seconds, that is 
received a time out. We denote the probability that the robot is timed out when executing 
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action a in state s as P{T = true\s, a) = a), and the probability that it is not 

timed out as P{T = false\s, a) = P~{s, a). While the first assumption (that the robot 
is where it intended to be after successfully executing an action a) is quite natural, the 
second assumption (that the robot in case of an unsuccessful action is where it started) 
is not. However, it reflects the fact that in case of a timeout the robot has wasted time 
without significant progress. 

The immediate reward of executing action a in state s also depends on the success 
of the action. For a successful execution, the agent receives the reward R~{s,a) = 
—l{a)/v{s, a) where l{a) denotes the length of the path to the target point of action a. 
The term v{s, a) denotes the expected average velocity for the execution of action a in 
state s. For a timed out action the robot gets an immediate reward of R'^{s, a) = — fto- 
In both cases the reward is an estimation of the time the robot loses when executing 
action a. In summary, the robot’s expected immediate reward is given by i?(s, a) = 
P"(s,a)R^{s,a). 

The Utility Function. The utility of a state s should reflect the time resources that are 
required to reach the destination from s. The closer s is to the destination, the higher is 
its utility. Because the state space S we consider is very complex, we cannot determine 
this utility U*(s) exactly. We therefore will use the following heuristic approximation 
of U*(s): 

Letp = [po,pi, ...,pn,] be the path from the robot’s current positions to its destination 
and l{p) be X)r=i \Pi-ijPi\- Further, let uavg be the robot’s average velocity while 
performing navigation tasks with its best plan execution policy. We can then approximate 
U*(s) as U*(s) = -l{p)/vavg. 



Action Selection. Using these simplifications we can replace equation l^by the follow- 
ing: 



7t(s) = argmax 
a^A 



E 

i6{ + ,-} 



P\s,a){R\s,a) + V\s,a)) 



( 2 ) 



where P+(s, a), P (s, a), and P (s, a) are as defined above and 



V (s,a) 



l(p) - 1(a) 
vavg 



U+(s,a) 



Kp) 

vavg 



( 3 ) 



To understand equation 0 recall our assumption that a successful action a leads 
the robot to the target of a, while an unsuccessful action leaves the robot where it 
started to execute a. The latter assumption biases the robot to prefer action with a higher 
probability of success. Please note that we have chosen the discounting factor 7 to be 1 
as the problem is a finite horizon problem where the agent always reaches an absorbing 
state after executing a finite number of actions. 

In order to get all information needed to solve the plan execution problem as the mdp 
derived above, the robot has to acquire for each state action pair (s, a) ( 1 ) the probability 
distribution P{T\s, a) and (2) the average velocity the robot will have when executing 
action a in situation s, v{s, a) if no timeout occurs. 
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3 Learning the Models 

Let us now consider how the function v and the action model P{T\S, A) can be learned. 
We will apply two alternative approaches: neural network learning and tree induction. 

Artihcial neural networks are well known as general function approximators. They 
have therefore often been used to approximate utility and reward functions in reinforce- 
ment learning. Tree-based induction methods 1BFOS84I . on the other hand, have the 
advantage that they provide in addition to a classification (in case of decision trees) or a 
value prediction (in case of regression trees) an explanation of the results they produce. 
The learned tree representations are often valuable resources for human inspection and 
automated reasoning. 



3.1 State Feature Vectors for Plan Execution 

Before we can apply any kind of learning algorithm we first have to define fhe feature 
language that is used to characterize the concepts and make informed predictions. In 
order to define an effective feature language we have to identify observable conditions 
that correlate with the navigation performance. 

In our learning experiments we have used the following features: (1) clearance to- 
wards the target position, (2) clearance at current position, (3) clearance at target position, 
(4) minimum clearance on path, (5) curvature of the planned path, (6) average clearance 
on path, (7) maximal minus minimal clearance on the path, and (8) relative length of 
path to target position. 

The robot’s clearance at any position is the distance to the next obstacle in its envi- 
ronment. The clearance towards the target position is the distance to the next obstacle in 
this direction, but relative to the euclidian distance to the target position. For all k points 
on the path to the target, we compute the clearance and keep the minimal clearance, 
the average clearance and the difference between maximal and minimal clearance as 
features. Another feature is the curvature of the path to the target point a which is l{a) 
as dehned above relative to the euclidian distance to the target point. To compute the 
relative length of the path towards a, we consider only those a on the path p such that 
l{a) is smaller than a constant Zmax (which is in our experiments was 800 centimeters) 
and the relative length of a as l{a) /Imax- 

Figure 13 illustrates the features (1) to (4) graphically. The features are relatively 
simple and can be computed efficiently. We believe that the use of more sophisticated 
features could probably further improve the robot’s performance. The automated learning 
of more expressive features will be subject of our future research. 

3.2 Neural Nets 

To learn the action model P{T\S, A) we have used a simple feed forward neural net 
with sigmoidal activation functions which was trained using an epoch back propagation 
algorithm. To speed up convergence we have normalized the features described above so 
that their normalized values are close to 0. Each normalized feature vector was associated 
with either 0 when no timeout was observed or 1 if a timeout was observed. The output 
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Fig. 2. The features (1) to (4). 



of the neural net (after a sufficiently long training) is a value in the interval [0, 1] which 
can he interpreted as the probability of a timeout. 

We have used the same neural network structure to learn the function v. Only the 
training examples differ. The output values in this case are: p = I’cur/i'max G [0, 1] 
where Vcur is the robot’s current, and Wmax is the robot’s maximal velocity. 

3.3 Tree Based Induction 

Decision trees (Sm53 can be interpreted as a set of production rules that are well 
understandable for humans. To learn the action model P{T\S, A) with a decision tree 
we classify each training example given by the set of features described above as true or 
as false depending on whether a timeout occured or not. In the decision tree framework 
a probability can be associated with each classification like this: if n is the number of 
training examples that are mapped to a decision tree leaf I that is associated with the 
classification c and m is the number of examples in this set that are classihed as c, we can 
associate the probability p = m/n with this classihcation (given the observed features). 

To learn the function is a regression and not a classihcation task. Regression trees 
IBPU!S84I are tree based function approximators and can be applied to regression tasks. 
Regression trees are similar to decision trees, but differ from them in that leaf nodes are 
not associated with classihcations, but with real values. A regression tree like a decision 
tree can be translated in a set of production rules. 

To build a regression tree a set of training examples associated with a node in the 
tree is split to minimize some given impurity measure. This impurity measure often is 
the empirical variance of the output value of the function to learn. For example, to split 
a set of n examples, S, with a variance in the output value cr^, into two sets and S 2 , 
a split is chosen that minimizes nial + n 2 <j| where rii is the number of examples in 
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Si and af is the variance in the set Si with respect to the output value. This process is 
iterated recursively until some given stopping criterion is met. 

We have used the following stopping criterion: For each split we have tested if the 
split reduces the variance significantly. If the best split does not reduce the variance 
signihcantly we stop growing the tree at that node. Whether a split reduces the variance 
signihcantly can be tested using a bootstrapping t-test lCnh9.5l . The only parameter of 
this test is a lower bound 9 of the estimated probability p, we will call significance level, 
that the split really reduces the variance. If p > 0 we say that the split reduces the 
variance signihcantly. The choice of 9 allows us to trade-off the prediction accuracy of 
the tree (on the training examples) against its complexity. 

Regression trees are limited in that they split the input space into regions with bound- 
aries that are parallel to the main axis of the input space. This limitation can be overcome 
by using multivariate splits. For our experiments however we have used a simple imple- 
mentation that was restricted to main axis parallel splits. 



4 Experimental Results 



In this section we will demonstrate (1) that the pol- 
icy dehned by equation El can be used to execute a global 
navigation plan quickly and reliably and (2) that the nec- 
essary models can be learned autonomously. We have per- 
formed both, simulator experiments and experiments on a 
real robot, to show that the learned action selection im- 
proves the robot’s performance substantially and signih- 
cantly. 

The experiments are run on an RWI B21 mobile robot 
platform and its simulator. For navigation we have used a 
re-implementation of the path planning module described 
in lTRR+9811 . which considers path planning as an MDR 
As reactive navigation component we have used the one 
described in IFRT97I . It generates possible trajectories for 
local navigation tasks based on simple models of the robot 
dynamics and chooses the trajectories with the highest util- 
ity. 

The setup of the experiments is as follows. After the 
learning phase, in which the robot has acquired the action 
models needed, a default and the learned plan execution 
method are compared. The default method chooses the next 
target point randomly between 1 and 8 meters ahead on the 
planned path. Roth methods are compared based on a sequence of k navigation tasks 
that is executed n times. We then test whether the learned action selection mechanism 
improves the robot’s performance signihcantly. This is done with a bootstrapping t-test 

mm . 
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4.1 Learning the Models 

To learn the velocity function v and the timeout probability P{T\S, A)we have generated 
some training and test data using the random action selection as described above and the 
simulator. We used a set of 5279 training- and 3266 test examples for the classification 
task (was the robot timed out when performing a given action?) and 4203 training- and 
2531 test examples for the regression task (what was the robot’s average velocity when 
performing an action?). This data volume corresponds to collecting data from robot’s 
runs that take about 24 hours. For the regression task we only considered examples where 
the robot was not timed out. 

In the experiments we will present here, we have not used new experience to incre- 
mentally improve the models mainly to simplify the evaluation of the experiments. 

Learning the Models with Neural Nets. For the classification task as well as for the 
regression task we trained a neural network with 8 nodes in the input layer, 8 nodes in 
the hidden layer and 1 node in the output layer. We performed epoch back propagation 
to train both neural nets. 

For the classification task we used a learning rate of 0.8 and a momentum term of 
0.9. After 203590 iterations (a night) we had 88.14% of the training- and 88.77% of the 
test examples correctly classified. 

For the regression task we used a learning rate of 0.8 and a momentum term of 0.95. 
After 406360 iterations (about 24 hours) we got an absolute error of 3.698 cm/s on the 
training- and of 3.721 cm/s on the test set where 60 cm/s is the maximal velocity of the 
robot. 

Learning the Models with Tree Induction. We used the same data to learn a decision 
and a regression tree. Table [Ogives the statistics of the training error, the test error and 
the number of generated rules depending on the significance level used in the stopping 
criterion for the decision tree learning. 



Table 1. The training error, the test error and the numher of generated rules depending on the 
significance level used in the stopping criterion for the decision tree learning. 



LEVEL 


TRAIN. ERROR 


TEST ERROR 


RULES 


0.5 


6.6% 


13.2% 


301 


0.6 


7.4% 


12.6% 


202 


0.7 


8.0% 


12.7% 


146 


0.8 


9.6% 


12.9% 


72 


0.9 


12.0% 


14.0% 


21 


0.95 


12.3% 


14.1% 


12 



Table Q gives the same statistics for the regression tree where the training and test 
errors are absolute errors in cm/s. 

In our experiments we have chosen the rules generated from the trees grown with the 
0.95 significance level. These rules are well understandable when inspected by a human 
operator. The trees are grown within about two minutes. 
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Table 2. The absolute training and test error and the number of generated rules depending on the 
significance level used in the stopping criterion for the regression tree learning. 



LEVEL 


TRAIN. ERROR 


TEST ERROR 


RULES 


0.5 


2.71 cm/s 


3.76 cm/s 


743 


0.6 


3.33 cm/s 


3.76 cm/s 


256 


0.7 


3.68 cm/s 


3.88 cm/s 


96 


0.8 


3.98 cm/s 


4.11 cm/s 


48 


0.9 


4.27 cm/s 


4.30 cm/s 


29 


0.95 


4.33 cm/s 


4.37 cm/s 


25 



4.2 Simulator Experiments 

In the simulator experiments we compared the learned and the random action selection 
mechanism based on a sequence of 14 navigation tasks in a university office environment 
that was executed 18 times. FigureO shows the set of tasks that had to be performed. 




Fig. 3. The set of target points that define the 14 navigation tasks performed in each iteration of 
the experiment (three of the points define two navigation tasks depending on the starting position 
of the robot). 



Experiment 1. In the first experiment we compared the performance of the two action 
selection mechanisms, where the models are learned using neural networks. The average 
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time needed to complete all tasks is 1 380 seconds with random action selection and 1107 
seconds with the learned action selection. This is a performance gain of 19.8%. The 
probability of a reduced average duration (computed with a bootstrapping t-test ICoh95l ) 
is 0.998, and the performance gain therefore clearly significant. The probability that the 
standard deviation in the robot’s behavior can be reduced when using the learned action 
selection mechanism is 0.98. Tabled summaries these results. 



Table 3. The results of Experiment 1 . 



average time with random action selection 


1380.44 s 


average time with learned action selection 


1107.78 s 


performance gain 


19.75 % 


probability of reduced average duration 


99.98 % 


probability of reduced variance 


97.53 % 



Experiment 2. In the second experiment we compared the learned action selection using 
the tree structured models with the random action selection. With the learned action 
selection the robot on average needed 925 seconds to complete the whole sequence, in 
contrast to 1380 seconds with the random action selection. That is a reduction of 33.0 %. 
The probability that the average time needed to execute the sequence of navigation 
tasks was reduced is 1 . Furthermore, the probability that the standard deviation of the 
execution times is reduced is 0.99. The results are summarized in TableHl 



Table 4. The results of Experiment 2. 



average time with random action selection 


1380.44 s 


average time with learned action selection 


925.39 s 


performance gain 


32.96 % 


probability of reduced average duration 


100.00 % 


probability of reduced variance 


99.03 % 



Experiment 3. A purely random action selection seems to be a quite weak standard 
for the performance comparisons. However, as we will show in this experiment, this is 
not the case. To demonstrate this, we compared a deterministic action selection with the 
random one: the robot always selects as next target point the last point on the path that is 
still visible from its current position. This is more or less the strategy that is used in our 
regular navigation system. Surprisingly, the performance is about 11.6% worse with the 
deterministic strategy compared to the random action selection. The robot in average 
needs 1541 seconds for the 18 navigation tasks compared to the 1380 seconds with the 
random action selection. Therefore the random action selection is significantly better 
than the deterministic one (with respect to a significance level of 99 %). However, the 
standard deviation is significantly smaller, 115.2 seconds compared to 237.9 seconds, 
with respect to the same significance level. 
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4.3 Experiments with the Mobile Robot 

In the experiments performed on the mobile robot, we test how well the models learned 
in simulation generalism when tasks different from those considered in the training 
phase have to be performed by a real robot. This is an interesting question because it is 
unrealistic to require state-of-the-art research platforms to perform the time-consuming 
learning without keeping them under surveillance. Therefore it should be possible to 
learn at least part of the models in simulation. 

The robot was to execute a sequence of 5 navigation tasks 1 8 times both with random 
action selection and with the informed action selection described above. The experiments 
were carried out in a populated university office environment. Figure^ shows the envi- 
ronment and the tasks that have to be performed within that environment. 




Fig. 4. The set of target points that define the 5 navigation tasks performed in each iteration of 
the experiment with the real robot. 

Experiment 4. When using neural nets to learn the models, the robot needed 362 
seconds in average to perform the five tasks, opposed to 420 seconds with the random 
action selection. This is a performance gain of 13.8%. The probability that the average 
time is reduced when using the action selection mechanism of equation|2|is 0.999. The 
probability that the standard deviation was reduced is 0.94. The robot’s performance can 
be summarized by descriptive statistics shown in Tabled] 

Experiment 5. In the last experiment we have compared the performance of the robot 
using equation 0 and decision/regression trees for the models with its performance 
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Table 5. The results of Experiment 4. 



average time with random action selection 


420.06 s 


average time with learned action selection 


362.00 s 


performance gain 


13.82 % 


probability of reduced average duration 


99.98 % 


probability of reduced variance 


94.59 % 



using random action selection: The average time needed to execute the sequence of 
five navigation tasks is 306 seconds with the learned models as opposed to 420 seconds 
with the random action selection. This is a reduction of 27.1%. The probability that 
the average time has been reduced with the learned action model is 1 and and the 
probability that the standard deviation was reduced is 0.99. The descriptive statistics of 
this experiment are summarized in Table 0 



Table 6. The results of Experiment 5. 



average time with random action selection 


420.06 s 


average time with learned action selection 


306.17 s 


performance gain 


27.11 % 


probability of reduced average duration 


100.00 % 


probability of reduced variance 


99.03 % 



4.4 Discussion of the Experimental Results 

The experiments show that the policy defined by equation |3 can be used to solve the 
plan execution problem and that the necessary models can be autonomously learned by 
the robot. In the domain, tree structured models perform slightly better than neural nets. 
However, these results do not support a general comparison of the two learning methods 
as we have used fairly simple implementation of both methods. 

A main problem for both learning methods is the noisy data. It is mainly caused by 
two facts: (1) The local navigation component has a lot of special purpose behaviors to 
react on exceptional situations, but the learning component currently can neither control 
nor even monitor their activation. This makes it rather difficult to predict the behavior 
of the reactive component and causes a lot of variance and noise in the training data. 
(2) The feature language is not expressive enough to fully discriminate situations where 
the robot is successful from those where it is not (partly because of (1)). We will address 
both problems in our future work. 



5 Related Work 

In this paper we have applied the MDP framework to the problem of navigation plan 
execution. To the best of our knowledge, this is a new application of the MDP framework. 
However, it has been applied to mobile robot navigation before, for example to plan 
navigation policies [KCK96 SK95fl . Reinforcement learning and especially Q-learning 
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have been applied to learning reactive navigation llThr96iRS97l . Often reinforcement 
learning is combined with neural network learning as in the case of IThr96L more seldom 
with the use of instance-based methods as in the case of IRS97II . 

More recently reinforcement learning has been combined with regression trees. Srid- 
haran et al. |:ST0()| applied Q-learning with regression trees in multi-agent scenarios. 
They point out that regression trees have been superior to neural networks both in the 
policies learned and the reduced training effort. Wang et al. IWD99I discuss the use of 
regression trees in combination with TD(A)-learning to solve job-shop-scheduling prob- 
lems. They utter the hope that regression tree learning might be suitable to become the 
“basis of a function approximation methodology that is fast, gives good performance, 
requires little-or-no tuning of parameters, and works well when the number of features 
is very large and when the features may be irrelevant or correlated”. 

Balac et al. iBUFOUl report the use of regression trees for (again purely reactive) 
robot control, but they do not compare regression trees to other function approximators. 
The use of decision trees - more specifically the use of confidence factors in decision 
tree leafs - for robot control is reported in |$A79BI . 

Our work is also related to the problem of learning actions, action models or macro 
operators for planning systems. Schmill et al. recently proposed a technique for learn- 
ing the pre- and postconditions of robot actions. Sutton et al. ISPS99I have proposed 
an approach for learning macro operators in the decision-theoretic framework using 
reinforcement learning techniques. 



6 Conclusion 



We have discussed how the problem of navigation plan execution can be formalized 
in the MDP framework and how the formalization can be heuristically simplified to 
make its solution feasible. This formalization allows for the application of standard 
learning techniques like neural networks and decision/regression trees. The approach 
has been tested both in simulation and on a RWI B21 mobile robot to demonstrate that 
it improves the robot’s behavior signihcantly compared to a plan execution method that 
selects actions randomly from a set of admissible actions. 

These results are important for the application of autonomous mobile robots to ser- 
vice tasks. Using the technique described in this paper we can build navigation systems 
that employ state-of-the-art global navigation planning and local execution techniques 
and let the robots themselves learn the heuristic, environment- and robot-specihc inter- 
face between the components. Even with a very restricted feature language and a much 
reduced action set we could achieve significant performance gains. The next steps on 
our research agenda are the development of a more expressive and adequate feature 
language, the introduction of more powerful actions such as turning in place, varying 
the speed, etc, and the development of more accurate approximations of the plan exe- 
cution problem for which the necessary models can be learned in moderate time (for 
regression/decision tree within a few minutes) and the necessary training data can be 
collected in a few days (in our case within 24h in simulation). 
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Abstract. This paper describes the DiKe model-hased diagnosis framework, 
which incorporates multiple diagnosis engines, multiple user-level system de- 
scription languages, a theorem prover, and a graphical user interface to provide 
an integrated toolset for the development of model-based diagnosis applications. 
The framework has been used for representing a number of application domains. 
We present the AD2L language, the main user language for the system geared 
towards use hy non-specialists, and discuss use of DiKe in various domains. 



1 Introduction 

Model-based Diagnosis and Model-Based Reasoning are two areas of knowledge-based 
systems research that grew out of the late 1980s’ disenchantment with traditional rule- 
based expert system technology. The goal was to avoid the brittleness of the latter sys- 
tems by using a Reasoning from First Principles approach, and the maintenance issues 
by providing high-level representation languages with unambiguous formal semantics. 
Overall that goal can be considered to have been attained as model-based systems are 
being employed in a variety of application areas. On the other hand, whereas rule-based 
tools are still widespread and used by many practitioners on actual applications projects, 
the model-based approach has so far not really moved out of the academic world. What 
applications there are are quite successful but still require the attendance of a research 
team to develop and implement system descriptions and implement or at least tune 
special-purpose reasoning engines. There is no widespread understanding of the prin- 
ciples, acceptance of the advantages, or support from a user community as with the 
continuing "grassroots" existence of various development environments for rule-based 
systems. 

Our goal is to facilitate the development of model-based diagnosis into a technology 
that can be readily used even by individuals without a formal training in AI techniques. 

* This work was partially supported by the Austrian Science Fund project N Z29-INF, Siemens 
Austria research grant DDV GR 21/96106/4, and the Hochschuljuhilaumsstiftung der Stadt 
Wien grant H-00031/97. 

** Authors are listed in alphabetical order. 
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To this aim we have built an integrated diagnosis toolkit that provides different types 
of diagnosis engines, graphical user interfaces, and user level languages for describing 
diagnosis knowledge but do not require detailed knowledge of formal logics. In fact the 
language AD2L was purposely defined to have an appearance similar to conventional 
programming languages that would help acceptance with engineers or software devel- 
opers. We describe the principles of AD2L, and then discuss the implementation of the 
framework, its use in domains as diverse as circuit diagnosis and software debugging, 
and ongoing work on the system. 



2 Model-Based Diagnosis 

Model-based diagnosis (MBD) irZHi.SII is a general approach to solve the problem of 
diagnosing malfunctions in technical, biological, or environmental I17llbll systems. In 
MBD a declarative model SD of the system is used to identify components COMP of 
the system that if assumed to be incorrect, cause the observed behavior OBS. Formally, 
diagnoses can be characterized in different ways, the most widespread being consistency- 
based I2B]: A set Z\ C COMP is a diagnosis iff 

SD U OBS U {^AB{C)\C G COMP \ Zl} U {AB{C)\C G A} 

is consistent. AB{C) indicates that a component C is behaving abnormally, and a cor- 
rectly behaving component C is described by ^ab{C). In general we want to compute 
diagnoses which are subset-minimal. 

The model must be compositional, i.e., provide behaviors of individual components 
from which the overall system is composed (such that the system description can be 
composed from the models of the components) but requires only to capture the correct 
behavior. The faulty behavior of components can be also incorporated into the MBD 
framework (see HI). The MBD approach is flexible and is not limited to diagnosis 
of physical systems, e.g., it has also been applied to solving configuration tasks 
and software debugging tTTCT . 

The main task of a MBD system is to determine components that are responsible 
for a detected misbehavior. In consistency-based diagnosis this is done by assuming the 
correctness of components and proving consistency of the given model and observations. 
If the assumptions lead to an inconsistency, they are called a conflict. Reiter’s hitting set 
algorithm uses the conflicts to compute all minimal diagnoses. Hence, diagnosis 

is reduced to search for all conflicts. Beside M the GDE ) makes use of this approach. 
Other MBD algorithms based on a form of belief revision O or on constraint satisfac- 
tion algorithms ITHTT . Most of the diagnosis algorithms utilize special data structures 
for search. 

Apart from theoretical work on MBD and modeling for MBD there are multiple 
applications of MBD described in the literature. In i:iM12 /l the authors describe a MBD 
system that operates the Deep Space One spacecraft. Other applications of MBD and 
model-based reasoning (MBR) are reported in 091.3.51 . For example, E3 introduces an 
MBR approach to nuclear fuel reprocessing, and describe the application of 

MBD in the automotive domain, a very promising area to apply MBD technology. 
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3 Building MBD Applications: The Problem 

An MBD application presupposes the existence of an implemented diagnosis engine and 
a model of the system that can be described using the language used by the diagnosis 
engine. The diagnosis engine makes use of the model and the given observations to 
compute (minimal) diagnoses. Most prototypical diagnosis systems tightly couple the 
diagnosis engine and the system description language which is used to describe the 
model. This has the advantage that there is no overhead on side of the modeling language, 
but has the disadvantage that models cannot be used by other diagnosis systems without 
substantial effort. What is required in order to solve this problem is a general system 
description language with well-founded syntax and semantics. Such a language must be 
capable of describing different kinds of systems from different domains. 

Although the use of a standardized and general system description language has its 
advantages, a general diagnosis framework should avoid too tight a coupling. Reasons 
are: (1) languages change, (2) in some applications it is better to use the basic model 
representation methods directly, (3) a general framework should be easily adaptable 
to other circumstances, and finally (4) the implemented diagnosis engine may not be 
capable to handle all aspects of the language because it is optimized for a given subset. 
Therefore, it is better to introduce a compiler that maps models described in a modeling 
language to the basic model representation methods provided by the diagnosis engine. 
The compiler has to ensure not only syntactical correctness but also the correct mapping 
of models to their corresponding representation. 

We propose the use of a general modeling language which allows for specifying 
not only the structure of a system and the behavioral models of the components but 
also additional diagnosis knowledge, e.g., fault probabilities, possible replacements and 
repair suggestions, observability of connections and states, correctness of components 
and component focus sets, logical rules stating physical impossibilities as described 
in m, and others. Every diagnosis engine that is capable of compiling the models 
written in such a general modeling language can make use of them. If using the proposed 
approach, we gain more flexibility, enhance model reuse, and focus the user more on 
modeling issues than on implementation issues. 



4 The DiKe Modeling Language and Implementation 

Our MBD application framework comprises two main parts: a modeling language 
(AD2L) and class library implementing different diagnosis engines. The modeling lan- 
guage allows specifying the behavior of components and the structure of systems. It is 
independent from the implemented diagnosis classes and could be used in other systems. 
Syntax and semantics of AD2L are well-defined. We have used the DiKe application 
framework in several different diagnostic systems. 

4.1 The AD2L Modeling Language 

The purpose of designing a dedicated system description language for model-based 
diagnosis is to support the user in writing the actual models. He should not be required 
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Fig. 1. Parts of a home power network 



to engage in applications programming, and the language should provide constructs to 
directly express the basic primitives that are generally used in system descriptions for 
model-based diagnosis. In other words, the language is supposed to provide a vocabulary 
that corresponds to the structure generally present in system descriptions for various 
domains. 

We assume a diagnosis model to be composed out of smaller model fragments . Such 
a model fragment describes the behavior of a single component, e.g., a n-input AND 
gate, whereas a complete model describes the structure and behavior of a whole system 
in a logical way. The art of writing model fragments is that of describing the behavior 
in a context independent way, i.e., the behavior description of a component should not 
determine its use. In practice context independence cannot always be achieved, nor is it 
possible to define a language that guarantees context independence. 

In this section we introduce the basic concepts of the AD2L language designed 
for the purpose of communicating diagnosis knowledge. Instead of formally describing 
the language we show its capabilities using an example from the electrical domain. 
Consider a home power network, which typically involves a connection to the local 
power supplier, fuses, sockets, and devices attached to sockets; lights, washers, and 
other power consumers. FigureQ]shows a small part of such a net. 

In order to write a model for a power network we (1) define types for connections, 
(2) declare a model fragment for every component, and (3) connect the fragments to 
receive the final model. 



Defining types. Types are used for representing the domain of connections and com- 
ponent ports. In AD2L there are 5 predefined types: boolean, character, string, integer, 
and real, with some predefined functions, e.g. H-,*, and others for integer and real values. 
In addition, the programmer can declare enumeration types. For example, in the power 
network domain we want to describe a qualitative model for currents and voltages, only 
using the information about whether a current or voltage flows or not. In this case we 
define the following type: 



type electrDomain : { “on“ , “off* }. 
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Apart from such simple enumeration types, AD2L allows the use of predicates and 
the specification of tolerances and equivalences. 

type quantDomain : real tolerance [ -5% , 10% ]. 
type myLogic : { ’0’ , ’L’ , ’1’ , ’H’ , ’X’ , ’Z’ } 
equivalence { ’0’ = ’L’ , ’ 1’ = ’H’ }. 

Tolerances and equivalences are used for determining a contradiction during com- 
putation. For example, if we can derive the value ’ 1 ’ for a connection S of type myLogic 
and we have an observation ’FI’ for S, then no contradiction arises. If no equivalence 
relations are defined, a contradiction occurs because it is assumed that a connection can 
only have one value. 

The use of predicates in type declarations is another feature of AD2L. Consider the 
case where a connection can have several values, e.g., a radio link that broadcasts the 
signal of several channels at the same time. The type for this connection is defined as: 

type channel : { “nbc“ , “cnn“ , “abc“ }. 
type radioLink : { predicate online ( channel ) }. 

The channel type enumerates all possible channels that can be broadcasted. A con- 
tradiction only occurs in this case if a connection of type radioLink has a predicate and 
its negation as its value at the same time, e.g., online(“abc“) and -online(“abc“). 

U sing types for connections has two advantages. The first is that type checking can be 
performed at compile time. The second is that the list of domain values can be employed 
at the user interface level to present a list of possible values, or for checking the validity 
of user input after data entry. 

Writing behavior models. The component declaration statement is the basic tool in 
AD2L for describing the interface and behavior of components. AD2L distinguishes 
between two different component declarations, atomic components and hierarchical 
components. Atomic components have a fixed, declared behavior and cannot be subdi- 
vided further. Hierarchical components derive their behavior from their set of internal 
subcomponents (and connections between them) which are separately described. The 
subcomponents themselves may either be hierarchical components or atomic compo- 
nents. 

Using the power net example, we now show the use of AD2L for writing atomic 
components. Verbally speaking, a light is on if its switch is on and it is connected to a 
current source. If the light is on, there must be a current flow and a voltage drop. Note 
that a voltage can be measured although there is no light and no current flowing through 
the bulb. If the bulb is broken, i.e., the component does not work as expected, then there 
is no current flow and the light is off. Formally, this behavior can be described in AD2L 
as follows: 

component light 

comment "This is a qualitative model of a light" 
input current, voltage : electrDomain. 
input switch_on : bool, 
output light on : bool. 
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default behavior nab 

Val(switch_on,true), Val(voltage,on) =:= Val(current,on). 

Val(current,on) =:= Val(light_on,true). 

Val(light_on, false) =:= Val(current,off). 

Val(light_on, false) =:= Val(switch_on, false). 

end behavior 

behavior ab 

=: Val(current,off). 

=: Val(light_on, false). 

end behavior 
end component 

In the first line of the AD2L declaration of the component light, a comment is given. 
It is followed by the declaration of the interface, i.e., the ports which are used for 
connecting different components via connections. The AD2L compiler checks the types 
of connected ports and reports an error if they are not equivalent. In our case we define 4 
ports : current, voltage, switch j?n, lighten. The declaration of interfaces allows to specify 
whether a port is an input or output port or both (inout). Note that this information is not 
used to restrict the behavior description. It is intended to be used by diagnosis engines 
to determine a focus set or to optimize questions to the user about values. In addition, in 
AD2L the programmer can specify parameterizable generic ports. A generic port can be 
used to configure the component for different systems. For example, a component with 
a generic number of inputs is defined by: 

generic Width : integer = 2. 

input i[l -Width] : bool. 

After the interface, the behavior of the component can be defined. It is possible 
to define several behaviors. Each of them has a name (also called a mode), e.g., nab 
standing for not abnormal. In the example we distinguish between two modes. One 
defines the expected and the other the faulty behavior of light. AD2L requires one mode 
to be designated as default mode. The default behavior is used by the diagnosis engine 
as a starting point for diagnosis. 

A behavior itself is described using rules. A rule consists of two parts (the left and 
the right side) separated by an operator =.■ or =. =. For rules of the form L =: R the 
semantics are easy: If L evaluates to true, then all predicates in R must be true. Rules 
of the form L =: = R are a shortcut for L =: R and R =: L. For rules of the form L =: R 
the left side is called condition and the right side action part (where the action simply 
consists of asserting the predicates on that side as true). 

The left and the right side of rules are conjunctions of predicates. Disjunctive sen- 
tences have no direct representation in AD2L for complexity reasons. Predicates are 
predefined. The use of quantifiers is possible. Note that this AD2L predicates are dif- 
ferent from data type predicates that are used as elements of a type and are defined by 
using the predicate keyword. Data type predicates are explained previously. The most 
important AD2L predicate is the Val predicate. Its first argument is the port and the 
second the value of the port. It evaluates to true if the port has the given value. Another 
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important predicate is Cond with a condition as the only argument. If the condition is 
true, the predicate evaluates to true. For example, the rule 

Val(anInput,X), Cond(X>20) =; Val(anOutput,true). 

specifies that if the value of aninput is greater than 20 the port anOutput must contain 
the value true. Note that Cond can only be used in the condition part of a rule. (Thus, in 
rules containing Cond the use of =. = is not allowed.) Another predicate is Fail which, 
if true, raises a contradiction. This predicate has no arguments and can only be used in 
the action part of a rule. Again, its use in =. = rules is not allowed. 

The use of quantifiers in rules is defined in AD2L. The intention is to use quantifiers 
for making the model as concise as possible. For example a quantifier can be used in the 
case we have to set all input ports to a specific value. 

=: forall INPUTS : Val(INPUTS,on). 

Note, that the existential quantifier (exists) can only be used in the condition part. 
In this case only the =.■ rule operator is allowed. The forall can be used in both parts of 
the rule. The quantification operator only influences the part of the rule where it is used. 
All of these restrictions are necessary to avoid complexity problems. 

The variable INPUTS is a built-in variable storing all input ports of the current com- 
ponent. There are several other built-in variables predefined in AD2L, e.g., OUTPUT 
and others. The user can also define variables using the variable declaration that must 
be located in the interface part of the component declaration. All variables are restricted 
to a finite domain. 

We define the semantics of quantifiers based on the semantics of rules and predicates. 

Forall Conjunctive sentences of the form forall X: P(X) op A (with |A| = n) are 
transformed into a single sentence P(vi), .... P(v„) op A, where Vi is an element of 
X and op is either =.■ or =. =. 

Exists Conjunctive sentences of the form exists X: P(X) =: A (with |A| = n) are 
transformed into a set of sentences P(vi) =: A,. . P(v„) =.■ A one for each element 
Vi of X. 

The user can extend the core behavior definition by additional properties, i.e., repair 
costs, actions, and probabilities, 
component light 

default behavior nab 
prob 0.999 
cost 2 

action "Replace the bulb" 

Val(switch_on,true), Val(voltage,on) =:= 

Val(current,on). 



As stated above, hierarchical components can also be defined in AD2L. Their decla- 
ration is discussed in the next section. We decided not to distinguish between hierarchical 
components and systems because there is no conceptual difference between them - both 
contain components and connections. 
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Fig. 2. The graphical representation of power jietwork 



Writing system models. Systems and hierarchical components consist of components 
and connections. Components can be either atomic components or again hierarchical 
components. The behavior of a system and a hierarchical component is given by the 
behaviors of the subcomponents. A hierarchical component can only have two behaviors. 
If it works correctly, all subcomponents are assumed to work correctly as well. The 
subcomponent behavior is given by their default behavior. In the other case, where the 
hierarchical component is assumed to fail, nothing can be derived. The probability of a 
hierarchical component C working correctly is computed using the probabilities of the 
default modes of the subcomponents {Ci, . . . ,Cn}' 

n 

p{nab{C)) = '^^p{defaultjmode{Ci)). 

i=l 

From the rules of probability theory follows p{ab{C)) = 1 — p{nab{C)). 

The user defines systems and hierarchical components by (1) declaring the used 
subcomponents, and (2) defining the connections between them. In our example the 
power net can be described at the system level as follows: 

component power_network 

input ext_voltage, ext_current : electrDomain. 

subcomponents 

fuse_l : fuse, 
socket. 1 : socket, 
light. 1 : light 

end subcomponents 

connections 

ext.voltage -> fuse.l(voltageJn). 
ext. current -> fuse.l(current.in). 
fuse. 1 (voltage .out) -> socket.l(voltageJn). 
fuse_l(current_out) -> socket. l(current_in). 
socket. l(voltage.out) -> llght.l(voltage). 
socket. 1 (current. out) -> light.l(current). 
end connections 
end component 

The graphical representation of the power jnetwork system is given in figure|3 
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Fig. 3. The Diagnosis Kernel (DiKe) 



4.2 The DiKe Framework Implementation 

The diagnosis kernel implements all classes and methods necessary for building a di- 
agnosis application, i.e., the class library for the user interface, the diagnosis engine, 
and the compiler. It was designed for flexibility and ease of use. The diagnosis ker- 
nel framework is implemented in Smalltalk (Visualworks 2.52 and 5i) and comprises 
generic classes for representing general interfaces and specific classes implementing the 
functionality. The portability of the Visualworks system has led to use of the framework 
under Solaris, Linux, and Win 95/98/NT. Figure Ogives an overview of the currently 
implemented parts. The diagnosis engine on the right is divided into a diagnosis system 
and a theorem proven The diagnosis system implements a diagnosis algorithm and stores 
knowledge about observations, connections, and components of a speciflc system. The 
theorem prover stores the behavior of the component to allow checking whether a system 
together with the observations and assumptions about the correctness of components is 
consistent or not. In cases where a consistency check is not necessary, a theorem prover 
is not used, e.g., the implementation of the TREE algorithm ED requires no explicit 
theorem prover. The implementation of Reiter’s hitting set algorithm CTTll on the other 
hand needs a theorem prover. 

Currently, our framework provides three different diagnosis engines. Two engines 
use Reiter’s algorithm while the other implements the TREE algorithm. Although the 
diagnosis algorithm is the same for the first two implementations, they use different the- 
orem provers. One uses a propositional theorem prover and the other a constraint system 
and value propagation. All concrete implementations have the same generic superclass. 
The generic diagnosis system class provides the interface, e.g., names of methods for 
executing diagnosis, requesting the next optimal measurement point, adding and re- 
moving observations, and others. The user writing an application using our diagnosis 
framework should choose the most appropriate diagnosis engine. If the model contains 
operations on numbers, the user should choose the value propagation algorithm. If the 
model is tree structured as defined in ED the user should take the TREE algorithm. In all 
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other cases the algorithm using the propositional theorem prover ensures best runtime 
performance that is almost equal and sometimes better than the performance published 
for other algorithms (mu. 

The diagnosis kernel provides two languages for describing specihc diagnosis sys- 
tems, e.g., a digital full-adder. The hrst language, DTalk is closely related to Smalltalk 
syntax and semantics. For every kind of diagnosis engine there are specihc language 
constructs representing the distinct behavior descriptions. While the knowledge about 
structural properties of a diagnosis systems are almost the same for every engine, this is 
not the case for the component models of DTalk. Therefore, we have developed a second 
more general language. This language AD2L has been described in a previous section. 
Models written in AD2L are not restricted to one diagnosis engine, although currently 
only the transformation of AD2L programs into the representation for the constraint 
based diagnosis engine is supported. 

Apart from classes for representing diagnosis knowledge, we have added classes for 
building user interfaces to the diagnosis kernel, to enable rapid prototyping of complete 
diagnosis applications. Using the demo applications and the diagnosis kernel classes as 
starting point, a hrst prototype of a diagnosis system implementing most of the required 
diagnosis functionality can be developed quickly. One of the demo interfaces uses a 
text-based user interface allowing to load systems and handle observations and other 
diagnosis knowledge, e.g., fault probabilities. The second variant uses a graphical ap- 
proach for representing components and connections, similar to a schematics editor. Both 
applications provide messaging interfaces for starting the diagnosis and measurement 
selection process. 

Diagnosis and measurement selection runtimes are competitive with other imple- 
mentations 112117171 . Parts of our VHDL debugger II UTIMOl l were implemented using 
the diagnosis kernel. 



5 JADE: A Debugger for Java Programs 

The DlKe class library has been used for several MBD projects. One of the most recent 
projects using the DiKe library is the Java Diagnosis Experiments (JADE) project. Dur- 
ing this project the MBD framework is used to implement a debugger for Java programs. 
We have developed two different models of Java programs. One abstract model 112 1 12011 
considers only the dependencies between variable occurrences in the program which are 
stored as propositional rules. The other model E28 represents the whole semantics of a 
(large) Java subset. This subset includes method calls, conditional statements, and while 
statements. This value-based model is represented as a constraint propagation system. 
Because of the different representations the implementation of the models makes use 
of different diagnosis engines. The abstract model is mapped to classes implementing 
the propositional theorem prover, whereas the value-based model is mapped to the im- 
plemented constraint propagation system. Both model implementations make use of the 
implemented hitting set diagnosis algorithm. 

The JADE debugger is a prototype system for research purposes and for demonstrat- 
ing the underlying model-based techniques. Development of the debugger was signih- 
cantly accelerated by making use of the available DiKe framework. First, no changes of 
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the basic classes of the DiKe library were necessary, we only needed to develop classes 
implementing the models. Because of available classes implementing similar function- 
alities and inheritance this was not a problem. Second, the standardized interface of the 
different diagnosis engines makes it easier to develop a graphical user-interface. Only 
small changes were necessary to adapt the interface of the dependency-based model to 
use it as an interface for the value-based model. Finally, the DiKe class library is very 
stable, because it has been tested on a number of examples and has been used for several 
prototypes so far. Because of the use of the DiKe framework the first Jade prototype 
could be finalized early in the project. The most expensive part for realizing the first 
prototype was the implementation of a Java compiler, the Jade interface, and the devel- 
opment of the models. As a consequence we were able to extend the debugger to support 
the whole Java language and to improve the user-interface which is very important. 

Other prototypes where we make use of the DiKe class library are a debugger for 
the hardware design language VHDL 111 II . a system allowing to interchange component 
models using TCP/IP socket communication, and a reconfiguration system for software 
parameters of a phone switching system iJ’i ll. all in the context of industrial projects. 

6 Results 

The DiKe MBD framework has been used to build prototypes for several different do- 
mains, e.g., debugging of VHDL designs 111 II . reconfiguration of software parameters 
of phone switching systems ||J'2||. audio routing systems, and more recently debugging 
of Java programs 1221251 . In all of these prototype applications the framework has been 
proven to be flexible enough and complete with respect to the provided functionality. 
The expressiveness of AD2L has been tested on several example systems. 

Besides providing a well designed framework for MBD applications, the improve- 
ment of diagnosis algorithms was also a goal of several projects in the past years. TREE 
and more recently TREE* is one of the outcomes of the projects that were integrated 
into the framework. In the following we compare the TREE* algorithm which is an 
extended version of the TREE algorithm, with El Eatah and Dechter’s SAB diagnosis 
algorithm [Q. Eigure0 gives the runtime results of TREE, TREE*, and SAB for tree- 
structured digital systems comprising And and Or gates as described in f2l. We see that 
both TREE and TREE* outperform SAB which was proven by [Q to be faster than 
GDE d and Reiter’s algorithm HSll . This holds especially for larger systems where a 
short runtime becomes an major issue. In [03 TREE and TREE* are described in detail 
and more empiricial results are given. 

7 Related and Future Work 

Since the beginnings of model-based reasoning several techniques for representing mod- 
els have been proposed. They mainly have in common that they are qualitative in nature, 
i.e., they do not use quantitative values. Such models are not only used in MBD but also 
in other fields. Eor example hardware designers speak about "low" and "high" or "true" 
and "false" instead of the exact voltage levels. In |4j an overview of qualitative modeling 
is given. Although the basic modeling principles seem to be established, there is almost 
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Fig. 4. Comparing TREE, TREE*, and SAB 



no accepted and widely used model description language available. Every reasoning 
system based on specific models uses its own languages. In addition, apart from M, 
where a WWW-based modeling system for sharing knowledge about physical systems 
is described, almost no work in the direction of providing tools for handling models 
and model libraries has been done. This system uses CML (Compositional Modeling 
Language) for describing models that can be translated to the Knowledge Interchange 
Format (KIF) |0- CML combines languages used for describing systems using Qual- 
itative Process Theory [(2| and the Qualitative Physics Compiler |0|. Other approaches 
for sharing diagnosis knowledge include UD where KQML Q is used as communica- 
tion language. Recent approaches for model interchange are mostly based on XML. We 
did not take this approach, because we consider XML to be primarily a language for 
information exchange, which does not provide support for defining semantics specific 
to modeling for diagnosis. On the other hand, it is straightforward to convert AD2L to 
an XML-based format. 

All previous approaches that rely on a logical description of the model are well suited 
for representation purposes. However, they are not so good when modeling is to be done 
by less experienced users. We face this problem in industry, where people are not familiar 
with the concepts of MBD and logic description languages (including Prolog). Although 
they see advantages in MBD compared to other approaches, they are sceptical concerning 
the realization of the advantages, e.g., reuse of models. Teaching students (especially 
from the electrical and mechanical engineering fields) the fundamentals of model-based 
diagnosis might alleviate the problem in the long term. A major step forward on the 
road to more general acceptance could be to uncouple the representation issue from 
the theoretical roots of the field and provide a dedicated representation that is more in 
line with the background of practitioners who might be "put off" by the appearance of 
pure logic. Advantages of a widely accepted language would include the possibility to 
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interchange models between researchers and companies, or between companies directly, 
the increase of reuse, and the certainty for companies that the model description can be 
used for a long time, thus saving the investments for modeling and providing an argument 
for using MBD. 

The language AD2L described in this paper is a proposal for such a modeling lan- 
guage. AD2L has been developed as part of a project with the goal of interchanging 
system descriptions over the Internet, and has been extended and adapted for industrial 
needs afterwards. The language definition is independent of the underlying diagnosis 
engine and provides language constructs directly representing model-based concepts, 
e.g., components and connections. In addition, other concepts from programming lan- 
guage design have been incorporated such as packages and strong typing. This allows 
for building model-libraries and avoids errors at runtime, which are central requirements 
of industry. 

Similar approaches have been considered in the past, such as the language CO- 
MODEL [6J, but have not found general use in industrial applications. AD2L on the 
other hand was developed in collaboration with industry. 

Although the introduced DiKe framework has been successfully used to implement 
different MBD prototype applications there are some open issues to be addressed. First, 
the class library contains different classes implementing the diagnosis engines and the 
AD2L language compiler. In our current implementation the AD2L programs are com- 
piled to a structure that can only be used by one diagnosis engine. A future implementa- 
tion should make a decision about which diagnosis engine to be used in order to optimize 
the overall diagnosis runtime. For example, a system that is tree-structured should be 
diagnosed using the engine implementing the TREE algorithm ED- 



8 Conclusion 

In this paper we have described an implementation framework for model-based diag- 
nosis systems that we have used in the last three years to implement systems as diverse 
as classical circuit diagnosis, reconfiguration of telecommunication networks, and a 
knowledge-based software debugger. The framework provides a graphical user inter- 
face, different diagnosis engines with different computational properties so that system 
performance can be adapted to the requirements and properties of a particular domain. 
It includes two different modeling languages, of which one, AD2L was specifically 
designed to provide a system independent platform for diagnosis knowledge base de- 
velopment and to be amenable to non-AI developers and engineers. 
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Abstract. A configurator using generative constraint satisfaction is presented. 
The configurator is applied for large EWSD telecommunication switches pro- 
duced by Siemens AG. The configurator has been in production use for more 
than four years and has required remarkably little maintenance effort. The 
knowledge representation with declarative constraints proved successful for 
modeling and solving large configuration tasks for real-world systems. 



1 Introduction 

Configuration is a challenging application domain for Artificial Intelligence tech- 
niques. The development of configurators is demanding for several reasons: 

• The task requires knowledge of experts. 

• The configuration requirements change frequently, leading to frequent changes in 
the component library and configuration constraints. 

• Configurator development time and maintenance time usually are short. 

• Users in different departments - such as development, sales, manufacturing, and 
service - often have conflicting goals as well as different automation needs, e.g. full 
automation of the configuration process. 

We show how we mastered those challenges with constraint satisfaction. 



2 Configuration Process 

The general configuration process with a knowledge-based configurator is shown in 
Figure 1. In [1] an overview of AI techniques for configuration is presented. 

In our application projects we use generative constraint satisfaction [2]. This AI 
technique proved suitable for mastering the challenges of large real-world configura- 
tion tasks. Generative constraint satisfaction employs a constraint network for problem 
solving which is extended during the configuration process. Generative constraints 
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hold for all components of a given type and are used as generators for extending the 
configuration with new components and connections. 



Component Library 
Racks 



Modules 

□ □ 

■ ■ 

Cables 

■ ■ 

■ ■ 

Software Modules 

□ □ 

Options 

□ □ 




Specification; properties of a 
correct configuration 



Select components 
Connect components 
Set parameters 









■^1 


— k 







Customer requirements 



Configuration with 
components, connections, 
and parameters 



Fig. 1. The configuration process uses a component library, a specification (set of constraints) 
and the customer requirements to compute a configuration of connected and parameterized 
components. Valid configurations are specified by constraints which guide the configurator in 
selecting components from the component library 



3 Modeling and Constraint Satisfaction 

Our constraint-based configurator needs only few types because properties can be 
inherited. During configuration the parameters can be adjusted to configure the mod- 
ules themselves. Consequently, knowledge bases for our configurator are smaller than 
for other problem-solving methods and can be maintained more easily. 

Valid configurations are specified with configuration constraints, for example 

• The first slot of every frame is reserved for power- supply modules. 

• The amount of electrical power consumed by all power-consuming modules in a 
frame must be less than or equal to the power output of the corresponding power- 
supply module. 

• The width of each frame must be less than 770 mm. 

• Analog and digital modules must not be mixed in a frame. 
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The capabilities of the underlying knowledge-representation scheme are crucial for 
developing and maintaining the configuration, as well as for implementing configura- 
tion-process tasks such as validating consistency and generating meaningful explana- 
tions. Consequently, our goal was to develop a knowledge-representation scheme 
general enough to be useful in a variety of application domains. 

Knowledge base. For configuration domains, component libraries describe the 
component types that can be used to build a configuration. We organize component 
types in an inheritance hierarchy that can be exploited during configuration. 

The constraint language for the knowledge bases of our configurator lets users ex- 
press all types of functionality, from that of key components to the formulation of 
complex customer-specified constraints. 

An attribute declaration consists of the attribute’s name and type (such as Num- 
ber, String, Enum, or Boolean), and, optionally, an initial (and backtrackable) or a 
constant value. During configuration, users must correctly set attributes for each com- 
ponent. They can do this manually or automatically using the configuration engine. 

Ports establish connections between two components. Each component type can 
have a set of port declarations, which consist of the port’s name, type, and domain. 

Constraints are a natural way to specify a relationship between attribute values or 
component ports. Constraints are always defined locally at a single component type. 
They can reference neighbor components, thereby navigating through the configura- 
tion. The declarative constraints also help users maintain the knowledge base: 

• The same constraint can be used to generate or check a configuration. 

• Constraints offer powerful representation and reasoning capabilities that can be 
extended to naturally express more complex configuration knowledge. 

• The simplicity and declarative nature of the basic constraint model let users define 
knowledge bases with precise semantics. 



4 Application 

Our configurator performs all core tasks in the configuration process. It automatically 
generates and expands the system, it modifies, checks, and explains the configuration. 
Users can manipulate the configurations interactively, regardless of whether configu- 
rations were just loaded or were generated by the configurator. 

Configuration outputs are primarily used to produce various part lists, assembly 
plans, and files for order processing and EWSD systems installation, including the 
cable plans. The configuration and reusable components are stored to serve as inputs 
for later use, such as existing-system expansion. Consequently, the configurator sup- 
ports many stages of the product life cycle, including sales, engineering, manufactur- 
ing, assembly, and maintenance. 
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Configurator for telecommunication. We have successfully applied generative 
constraint satisfaction for configuring the EWSD (Elektronisches Wahlsystem) digital 
switching systems developed and manufactured by Siemens AG. We implemented it 
[3] using our domain-independent configuration tool, COCOS (configuration by 
constraint satisfaction). 

With the configurator the sales engineers produce large EWSD configurations that 
consist of approximately 200 racks, 1,000 frames, 30,000 modules, 10,000 cables, and 
2,000 other units each. The knowledge base comprises approximately 20 types of 
racks, 50 types of frames, 200 types of modules, 50 types of cables, and 50 types of 
functional units for controlling the switching system. 

Concerning development costs and functionality of use, e.g. flexibility for the user 
during the configuration session, we compared our new configurator with two earlier 
configurators. The old configurators had been developed with non-AI software devel- 
opment methods and tools. 

Numerous changes to the EWSD system triggered a redesign of the program code 
of the old configurators, resulting in significant maintenance effort. In our new knowl- 
edge-based configurator minor changes to the knowledge base could cover all these 
requests. In addition, the knowledge base is more compact than traditional software 
code. Changes in the product library that trigger a change at multiple points in the old 
configurators cause only a single point of change in our new configurator. 

Maintenance costs over several years depend on the technology of the configurator 
kernel and on the user interface design. During 4 years of its production use the main- 
tenance effort per year of our configurator for EWSD switches amounted to between 
5% and 10% of the initial development effort, which is very efficient. 

5 Conclusion 

Our application of generative constraint-satisfaction techniques in the configurator for 
EWSD switches shows that expressive knowledge-representation languages can be 
successfully used in a production environment for large, complex domains. Using our 
techniques proved successful in terms of cost and time. It also enhanced functional 
capabilities. Such knowledge representation can contribute greatly to improve the 
maintenance of knowledge-based systems. 
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Abstract. Offering an individually tailored service to passengers while 
maintaining a high transportation capacity of an elevator group is an 
upcoming challenge in the elevator business, which cannot be met by 
software methods traditionally used in this industry. 

AI planning offers a novel solution to these control problems: (1) by 
synthesizing the optimal control for any situation occurring in a building 
based on fast search algorithms, (2) by implementing a domain model, 
which allows to easily add new features to the control software. 

By embedding the planner into a multi-agent system, real-time inter- 
leaved planning and execution is implemented and results in a high- 
performing, self-adaptive, and modular control software. 



1 Customization and Efficiency Challenge Innovation 

As many other industries, elevator companies are facing two main challenges 
today: (1) the pressure on building costs requires to improve the transportation 
capacity of elevator installations, (2) increasing competition challenges diver- 
sification and mass customization strategies, which aim at providing new and 
individually tailored services to passengers. 

Schindler Lifts Ltd. has developed the Miconic-10^^ elevator destination con- 
trol where passengers input their destination before they enter the elevator. The 
Miconic-10^''^ control has been introduced into the market in 1996 and since then 
more than 100 installations have been sold worldwide. A 10-digit keypad outside 
the elevators allows passengers to enter the floor they want to travel to. After 
input of the destination, the control system determines the best available cabin 
and the terminal displays, which elevator the passenger should take. The current 
Miconic-lO^'^^ control uses a rule-based, heuristic allocation algorithm. It groups 
passengers with identical destinations together and thereby allows them to reach 
their destination faster and more comfortable. 

By using identification devices such as smart cards, pin codes, or WAP 
phones, passengers can even be recognized at an individual basis and more in- 
dividually tailored services can be implemented: access restrictions of certain 
passenger groups to certain zones in the building, VIP service to transport some 
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passengers faster than others, e.g., medical emergency personal, and the separa- 
tion of passenger groups that should not travel together in the same elevator. 

Today, elevator systems can offer these services only in a very limited way by 
permanently or temporarily restricting the use of selected cabins. It is impossi- 
ble to integrate and embed these functionalities directly into the usual normal 
operation of an elevator group and the restricted usage of some of the cabins 
dramatically reduces the transportation performance. 

The work summarized in this abstract has been driven by a rigorous formal 
approach. In the early studies in 1998 and 1999, the complexity of the problem 
has been investigated and it has been proven that even in the simplest case, des- 
tination control is NP-hard 0. Based on these results, a comparative analysis 
has been conducted, which modeled destination control as a planning problem, 
as a scheduling problem, and as a constraint satisfaction problem 13 EH . Mod- 
eling the problem from a planning perspective seemed to be the most natural 
approach Etnd resulted in the miconiclO domain, which has also been used in 
the AIPS-00 planning systems competition. The formal modeling of relevant do- 
main properties in PDDL allowed to precisely define the services and prove the 
domain-specific algorithm to be sound and complete. It also helped in generating 
test problems that were used to verify the implementation. 

There are two aspects of the problem: The static, offline optimization prob- 
lem for one elevator, which requires to compute an optimal sequence of stops 
for a given, fixed traffic situation in a building. The dynamic, online decision 
problem, which needs to cope with the immediate and unknown changes of traffic 
situations. 



2 The Offline Problem: A Case for AI Planning 

The offline optimization problem for one elevator is given by a particular traffic 
situation in a building: the state and position of the elevator, the unanswered des- 
tination calls of passengers who are waiting in the building, and the destination 
calls that have already been picked up and where passengers are currently trav- 
eling in the elevator towards their destination. Given a number n of destination 
calls with board floor b and exit floor e as (6i, ei), (62, 62), (63, 63 ), . . . , (6„, e„) 
we need to compute a totally ordered sequence of stops si, S 2 , ■ ■ ■ , Sk such that 
each Si corresponds to a given board or exit floor (no unnecessary stops should 
be contained in the sequence) and where each bi is ordered before each Ci (since 
passengers have to be picked up first and then delivered to their destination). 

In practice, one is interested in finding stop sequences that minimize a given 
optimization criteria, e.g., minimizing the average waiting times of passengers 
or minimizing the overall time the passengers spend with the elevator from the 
moment they insert the call until they reach their destination. 

The domain specific planning algorithm constructs a sequence of stops for a 
single elevator out of the stop actions that are applicable in a given situation. 
The following properties turned out to be key to success: (1) the state represen- 
tation and the choice of data structures to implement it, (2) the search algorithm 
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as a combination of a depth-first, branch-and-bound search with forward check- 
ing techniques from constraint reasoning, (3) an admissible heuristic function 
that estimates the distance to the goal state in such an effective way that the 
branching factor in the search space is reduced by 60 to 90 %. The planning 
system allows to find optimal plans of up to a length of 15 to 25 stops in less 
than 100 milliseconds even in search spaces containing between 10^^ and 10^® 
states, which are frequently generated by data from real buildings with high traf- 
fic peaks. The planning system is able to deliver optimal plans given the tight 
real-time requirements because it works on the level of destination commands 
and returns an abstract sequence of stops. The execution of these plans requires 
to translate the stop sequences into the much more fine-grained level of elevator 
control commands. 

3 The Online Problem: Real-Time Interleaved Planning 
and Execution with Failure Recovery Mechanisms 

For each elevator, a so-called jobmanager has been developed, which controls 
a single cabin with its drive and various doors. A jobmanager is a holon of 
agents responsible for different tasks in the control, see Fig. Q1 The agents com- 
municate via an asynchronous messaging system supporting publish/subscribe 
mechanisms and allowing a peer-to-peer communication between the control and 
hardware components such as drives, doors, and terminals. 



Jobmanager 




Fig. 1. The jobmanager as a multi-agent system implementing interleaved planning 
and execution for a single elevator. 



The broker receives the offer requests from the terminals and adds the new 
calls to the world model, which is part of the initial state representation for the 
planner. It initiates the planning process and evaluates the returned plan. Based 
on the evaluation, it sends offers to the requesting terminals, which book pas- 
sengers via an auctioning process. The car driver is responsible for executing 
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the plans. Given an abstract sequence of stops, the car driver maps these into 
a fine-grained temporal sequence of activities, e.g., accelerating, moving, land- 
ing, opening doors, etc. The observer updates the world model of the planner 
solely based on information that it receives from the doors and the drive. This 
model updating independently of the car driver’s actions is very important to 
keep world model and reality in accordance. The failure recovery maps the 
activities of the car driver to the information it receives from drive and doors 
and verifies whether the planned activities are correctly executed. It implements 
a very flexible approach to deal with hardware failures, situations that made 
world model and reality fall apart, and passengers who behave not as assumed. 
The configuration manager provides information about the building layout, 
i.e., the number of floors, access zones, passenger groups and access rights, active 
services, etc. 

Each component is a self-acting agent that initiates activities when certain 
events occur. This can trigger several agents simultaneously and their activities 
can run in parallel. The distributed control is able to deal with such interfering 
events. 

The developed control software has achieved two major results: First, we 
obtain a much more modular and compact code comprising only 8000 lines in an 
object-oriented language. The underlying agent-based architecture leads to very 
clear interfaces and allows it to further develop the different agents independently 
of each other. The planning algorithm is able to unify the software for various 
elevator platforms, e.g., cabins with multiple decks. 

The situation-dependent optimization improves the transportation capacity 
of elevator systems significantly. The currently used heuristic allocation algo- 
rithm already improved performance by 50 to 80 % when compared to conven- 
tional control systems where passengers press the destination buttons inside the 
cabin after they have been picked up. With the intelligent planning system, per- 
formance is increased by another 10 to 30 % depending on the traffic pattern. 
Average waiting times remain roughly identical, but travel times within a cabin 
get reduced by up to 35 %. 
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Abstract. We present a navigable semantic network, that can be calculated 
automatically from document database, as an important component of a 
knowledge management portal that was developed for a corporate intranet. 
Users of the portal can navigate through a semantic network of terms, that helps 
users to determine the company specific meaning of terms. The semantic 
network utilizes the standard search engine of the portal and offers a query 
expansion mechanism for unknown terms, that retrieves only documents with a 
company specific default meaning. 



1 Introduction 

The terminology found in corporate intranets forms a mini-universum, which might 
differ significantly to an everyday life usage of that words and terms. In corporate 
intranets, one finds very specific meanings of words, that these words typically do not 
show in a non specialized context. For example in banking industry the term „Risiko“ 
(risk) is used in a lot of very specific ways, that comprise a variety of bank specific 
kinds of risks. On the other hand, there are terms that have a rather well defined 
default meaning in an every day life context that are not likely to he found with this 
non-specific meaning in the document-hase of a specialized industry. In addition, a 
company’s terminology contains meanings for words, that are completely different to 
the meaning these words have normally. In Dresdner Bank, e.g., there is a very 
important application called „Bingo“, which has nothing to do with the favorite game. 
Typically, in corporate terminology, there exist a big group of terms, whose meanings 
are only known to specialists. 

This situation creates a very difficult situation for gathering information from 
corporate documents knowledge bases like, e.g., a corporate intranet, when a user just 
sends a single term without additional information to a search engine. Unfortunately, 
the lack of additional information, is in a lot of cases the very reason for questioning a 
search engine. 

Thus, satisfying the information needs of an employee depends on finding a 
specific meaning of rather general terms, or a company’s specific meaning of seldom 
terms, which are unknown to the user. 
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In addition, gathering information in a business context means that it is important to 
identify the business process or the process owner that are correlated with a term. The 
identification of proper names of people that are responsible for a process or a product 
is often the most relevant result for the information need that is behind a user’s query. 

Standard search engines often can not meet this requirements, since they deliver 
whole documents and can not identify correlations between terms, proper names and 
names for processes, which are crucial for the understanding of a company’s 
organization. 

In this paper, we describe a component of a knowledge management portal for the 
corporate intranet of the Dresdner Bank. The tool visualizes term in the neighborhood 
of their most salient contexts. The relations between terms are displayed in a semantic 
net structure, in which the user can navigate through the company’s individual 
conceptual structure of terms, people and processes. 



2 The Architecture 

The intranet of the Dresdner Bank contains more than 400.000 documents, most of 
them HTML-pages, documents in Microsoft Word format, Microsoft PowerPoint 
presentations and documents in PDF format. There is a standard search engine 
available that indexes these documents and offer standard search functionalities. In 
the design phase of the knowledge management portal, we decided on three main 
topics. 

First the portal’s information offering was designed on the base of an empirical 
model of the users’ information needs. We used a web-data mining approach, where 
we analyzed the users’ questions to the existing search engine and cluster them in 
groups of common information needs [1]. 

Secondly, we decided not to develop a new search engine for the retrieval of 
documents, but use the existing search engine on the top of a query-expansion 
mechanism feed by the semantic net component. 

For the storage of the analytical data, we decided to use a standard relational 
database, that forms what we called a universal linguistic index (ULI) for the data. 
Performance optimizations and interfaces could be programmed using the database 
API, which guarantees professional support and a standard connection to the banks IT- 
infrastructure. 

The systems consists of three modules: A data gathering component (a so-called 
„spider“) that crawls the intranet, filters the different document formats and 
normalizes the documents by tokenization and a sentence-based segmentation. The 
second module is a linguistic analysis component that extracts named entities and 
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identifies syntactically licensed relations between terms. Using a simple anaphora 
resolution, the system is able to find semantic relations between terms that occur in 
different sentences by a simple coindexing mechanism. 

The third component is a relational database that stores the output of the flat 
linguistic analysis the word and relation counts together with a document’s meta- 
information like, e.g., the author of the document and its creation date. 



3 Automated Compilation of Term Correlation Graphs 

Big corpora contain a lot of implicit knowledge about the meaning of words in terms 
of the contexts a word typically occurs with. Knowing the most important contexts in 
which an unknown term is used, often yields enough information to determine the 
meaning of the word itself. Correlation analysis of words have quite a tradition in 
statistic computational linguistics and are often utilized in advanced information 
retrieval, e.g. [3]. Typically, a correlation measure between two words is calculated by 
counting the two words’ overall occurrences and the number of common occurrences 
of both words in a document, a paragraph or a sentence. If two words tend to occur 
significantly more often than by chance, they can he assigned a positive correlation 
measure. Measures that have been proposed are chi^, mutual information and others 

[4]. 

Although the algorithm for the calculation of a correlation matrix between all terms 
in corpus is rather simple, there are some difficulties that reduce the effect of the 
correlation matrix for the purpose of, e.g., query expansion in information retrieval. 
An important question is, how to define the window in which co-occurrences between 
words are selected. Taking the whole document often yields a lot of accidental 
correlations, which are hard to filter out against seldom but significant correlations. 
Looking only at the sentence level misses a lot of counts for correlation which are 
often expressed beyond sentence boundaries by anaphoric constructions. In addition, 
not each co-occurance of two terms within a sentence, means that these words are 
necessarily semantically related. 

To yield a precise sample of relations, we only count pairs of words, which are 
licensed by a set of predefined syntactic configurations. We used a simple regular 
grammar for German that yields a rather high precision, which we prefer in this 
situation over a high recall. The tradeoff of this approach is, that one yields even on 
corpus with over 40 million words, a lot of counts of pairs that are only observed two 
or three times, even if there exists a rather well known semantic relation between the 
terms. 

We therefore used a two step algorithm, that first calculates the correlation between 
all pairs of words. In a second step, we use a clustering algorithm to determine 
correlation that are not directly visible in the sentences of the corpus but could be 
detected by comparing the context vectors of two terms. We call the first type of 
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correlation the syntacmatic correlations and the second type the paradigmatic 
correlations. 

As an example for the importance of a paradigmatic relation, we observed in our 
intranet corpus a correlation between „text-mining“ and „speech-technology, because 
both terms share the same people and the same department as a common context, even 
if there is no sentence, where a direct relation between both terms can be found. 

The result matrix of correlated terms is calculated as the weighted sum of both 
correlation values for each pair. While the first step can be calculated very efficient, 
since we use a threshold on the counts of the pairs, the second step has a quadratic 
complexity, since each term with all its contexts must be compared with all other 
terms and their contexts found in the corpora. After optimizing the database access 
and using thresholds on the term count, we were able to calculate the correlation 
matrix for the most frequent 50.000 words of the corpora in seven hours on a moderate 
Pentium computer with half a gigabyte RAM. 

The matrix is visualized as a graph - or semantic network - with weighted edges 
according to the syntacmatic, paradicmatic or a overall correlation values. A user can 
type in a term, while the systems calculates all nodes with distance two and displays 
the resulting network using a Java applet GUI. The response time of the system is with 
less than a second for a query reasonable fast. Now the user can navigate the net by 
clicking on nodes to expand them, or directly search for a term, utilizing the search 
engine. 

We add a so-called topic detection mechanism, which uses the semantic net as the 
basis for a query expansion mechanism. Thus, selecting a term for topic detection in 
the semantic network’s GUI, triggers a query expansion mechanism that selects a 
number of neighbor nodes in the net with the highest correlation with the terms and 
sends this set of pairs (which are connected with a NEAR operator) as a conjunction to 
the search engine. We use a kind of constraint relaxation mechanism, when the search 
engine does not find a result for the whole conjunction. 
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Abstract. With the advent of web technologies the supply logistics in industry 
received a boost in what is nowadays called eSupply, Supply Chain Manage- 
ment, or - even more appropriate - Supply Net Management (SNM). In this 
contribution research in this domain is motivated by sketching the complexity of 
SNM especially in automotive industry and by showing up possible pathways 
into a visionary future. Then essentially two current research tracks in Daimler- 
Chrysler R&T are addressed, namely (1) supply net configuration, simulation 
and assessment, and (2) distributed supply net planning. In both domains agent 
technology is used, partly together with other AI techniques. 



1 Context and Complexity of Supply Net Management 

These days car manufacturers are changing their businesses rapidly and radically. This 
is true both for the classical value chains beyond car manufacturing (e.g., distribution, 
after sales services etc.) and for the pre-assembly value chains, i.e., in the supply and 
procurement business [1]. 

In the center of an automotive 0E1V|] are the assembly plants. These are provided 
on the first-tier level both by plants belonging to the OEM itself (e.g. engine plants) 
and by independent companies, the system suppliers. In principle this net goes up to 
the raw material. On the downstream side the assembly plants serve regional markets, 
retailers, and eventually the customer of the car, who is essentially pulling at the sup- 
ply net. 

To give an impression of the complexity of supply net management in Daimler- 
Chrysler automotive: more than 3000 suppliers (already 130 on the first-tier level), 
high degree of product complexity (about 10.000 parts/car), high degree of customiza- 
tion (esp. for supply parts), high volume production rate (up to 2.000 cars/ day and 
plant), long cycle times in some chains due to long lead times, various time and space 
constraints (w.r.t. capacity, stock size, part and component size). 



* This contribution has more than one “father” in my research team. I especially thank Hartwig 
Baumgartel and Jbrg Wilke for their imagination and thorough work. 

^ Original Equipment Manufacturer, the driving element in the value chain 
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Already very simple supply chains exhibit a surprising dynamics. So for instance in 
the MIT beer game [2], which consists of a linear chain with only four participants, 
simple changes in the demand of the consumer lead to rather unexpected stock and 
production policies at the nodes further down the chain. This demonstrates the bull- 
whip effect: forecasts and actual demand differ the more one goes down the chain. 

Supply net management, has to cope with (1) A-synchronism: Effects related to 
decisions occur with long time-lags. (2) Non-locality: Effects of local decisions (often 
made with incomplete knowledge) occur at places far away from the point of decision 
making and are not visible at other decision points (3) Non-linearity: The net effect of 
the many parameters in the supply net on common objectives is not predictable. There 
are positive and negative feedback loops. Most quantities are bounded (capacities, 
batch sizes, safety stocks, buffers, ...) 



2 From Isolated Enterprises to Agile Supply Communities 

The current situation in the inter-related world of suppliers, manufacturers, sales, and 
customers is characterized by isolated planning islands. Transactions are treated with 
only minor responsiveness, without proper consideration of demand, and control is 
largely reactive. The information exchanged (e.g. by EDI) is semantically poor, and 
mainly limited to the transactional level. 

One may envision a world [3] in which knowledge/content is exchanged (forecasts, 
orders, availability, shipments), where planning and optimization happens within the 
net, and where plans are adjusted and re-optimized in real-time. In this world a supply 
net is designed and operated for win-win situations of the OEM and its suppliers, 
change management and trouble shooting are utmost agile. There is order visibility in 
the sense of permanent evaluation and adaptation of capacity and delivery plans ac- 
cording to the current state of customer orders, production progress, inventory, and 
supply possibilities. This applies also downstream to the customer: She has the op- 
portunity to use the delivery date as a configuration restriction of a vehicle (“config- 
ure-to-promise”), and she becomes informed by the dealer if problems arise in the 
production process, making transparent the decision alternatives. 

This collaborative world can only be realized by (1) distributed SNM (2) integra- 
tion of APS0 systems across company borders, (3) inter-organizational supply net 
logistic control, (4) VP1S0 based planning data exchange. These requirements cry for 
AI, especially agent technology, since a supply scenario has all the characteristics for 
an agent system: It is distributed, with subsystems having a high degree of autonomy, 
and the interactions among the subsystems and with the environment are highly dy- 
namic. However, agent technology alone is not the key to open the door to future 
SNM. As it is often the case for real-world applications, agent technology is to be 
combined with both other innovative techniques from Al (constraints, uncertainty 
reasoning, ontologies, data mining, ...) and with techniques from established logistics. 



^ Advanced Planning System, e.g. supply chain planning software like SAP/APO 
Virtual Private Network 
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3 Supply Net Configuration, Simulation, and Assessment 

Our work in this domain aims at measuring the performance of collaborative sub-nets 
within the complete supply net. As of today, it is completely unclear, which kind of 
information exchange is appropriate for a good performance of the net. The essential 
logistic information resides in the BOM0 and one of the decisive questions of a well- 
performing automotive supply net is “who does the BOM-explosion at which stage at 
which time to which granularity ?” 

We try to find an answer to this question by simulation. The technical approach is 
to model supply net nodes and internal functions as software agents, each of them 
representing distributed planning domains. By this also proprietary and confidential 
data can be encapsulated. There are different agent types, each type characterized by 
specific patterns of behavior adjoined with specific sets of variables that are in the 
responsibility range of this very agent type. After being modeled, the dynamic simula- 
tion of the net receives e.g. demand functions and the output are supply net objectives. 
The simulation results then give hints how certain internal parameters of the net can be 
tuned to result in a leverage for supply net objectives. 

The basic idea of this approach originates from the DASCh-System [4], which 
allows to configure and to simulate generic, linear supply chains. Together with 
ERlM/CEC|]we extended this to a system which allows to simulate nets, and which 
respects certain requirements met in car industry. We were already successful in ex- 
hibiting qualitative process improvements [5]. By a further extension from the mate- 
rial flow to the logistics control level (which amounts to adding a further agent type) 
we are currently about to quantify the financial benefits for the partners involved. This 
is the foundation for inter-organizational supply net performance management. 



4 Distributed Supply Net Planning 

In principle it is imaginable that all relevant information about the status of a supply 
net is collected in a central data base, and that planning and optimization happens on 
this central unit. However, since independent enterprises are involved in SNM, it can- 
not be expected that all of them are willing to provide their data (exhibiting more or 
less explicitly also their policies) to a central instance. A company like Motorola for 
instance, being a supplier for Bosch, being a supplier for Mercedes-Benz, can hardly 
be motivated in displaying its planning data. 

We follow an approach which explicitly respects local authority and responsibility. 
This was already successfully applied to capacity balancing over independent units in 
an assembly plant (so to say an OEM internal supply net [6]). In our approach we start 
from the consumer demands. These are propagated upstream in the net, and because of 
the network structure they can possibly be met in different ways. Depending on local 



^ Bill of Material 

® ERIM/CEC is a research organization residing in Ann Arbor, Michigan. 
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capacities conflicts may arise. We model the units in the net by agents, which commu- 
nicate upstream and downstream about their capacity constraints, possible constraint 
violations and their origin and, if it exists, find a solution for the demand requirements, 
or allow to track down hard capacity conflicts and trace their propagation. 



5 Conclusion and Outlook: Collaborative Business 

We believe that next generation eBiz is cBiz (collaborative Business). This is not only 
a new buzzword, but indicates a paradigm shift in logistics, in the sense that partners 
in the supply net will re-consider their roles and recognize that only by collaboration 
the win-win potential inherent in innovative supply net solutions can be realized. 

Collaborative business has both technological and non-technological aspects. 
From the non-technological point, aside from the willingness to organizational 
changes, a feeling of trustful partnership must evolve. On the technology side we 
foresee three essential techniques. At first we believe that agent technology (for which 
collaboration is more or less inherent) is very appropriate to model, plan and run a 
supply network. And since at the very end everything boils down to bounded capaci- 
ties at the different sites, constraint technology should be utilized. An important fur- 
ther strand are ontologies: The more semantically rich the context of SNM is de- 
scribed, the better is the quality of information exchanged by the supply net partners, 
and the more efficient is the collaboration. 
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